Library Dependencies

Learn how to declare and manage library dependencies in sbt projects

This guide explains how to declare library dependencies, understand dependency operators, configure resolvers, and manage dependency conflicts in sbt.

Declaring Dependencies

Dependencies are declared using the libraryDependencies setting:

build.sbt

libraryDependencies += "org.typelevel" %% "cats-core" % "2.10.0"

The += operator adds a single dependency, while ++= adds multiple dependencies.

Dependency Operators

sbt provides several operators for declaring dependencies:

The %% Operator (Scala Binary Version)

The %% operator automatically appends the Scala binary version:

build.sbt

// With scalaVersion := "2.13.12"
libraryDependencies += "org.typelevel" %% "cats-core" % "2.10.0"
// Resolves to: org.typelevel:cats-core_2.13:2.10.0

// With scalaVersion := "3.3.1"
libraryDependencies += "org.typelevel" %% "cats-core" % "2.10.0"  
// Resolves to: org.typelevel:cats-core_3:2.10.0

Use %% for Scala libraries that are compiled for specific Scala versions. This ensures binary compatibility.

The % Operator (No Version Suffix)

The % operator uses the exact artifact name without appending Scala version:

build.sbt

libraryDependencies += "com.google.guava" % "guava" % "32.1.3-jre"
// Resolves to: com.google.guava:guava:32.1.3-jre

Use % for Java libraries or when you need exact artifact control.

Multiple Dependencies

Declare multiple dependencies using ++=:

Sequential

libraryDependencies ++= Seq(
  "org.typelevel" %% "cats-core" % "2.10.0",
  "org.typelevel" %% "cats-effect" % "3.5.2",
  "co.fs2" %% "fs2-core" % "3.9.3"
)

Configuration Scoping

Dependencies can be scoped to specific configurations:

build.sbt

libraryDependencies ++= Seq(
  "org.scalatest" %% "scalatest" % "3.2.17" % Test,
  "ch.qos.logback" % "logback-classic" % "1.4.11" % Runtime,
  "org.scala-lang" % "scala-compiler" % scalaVersion.value % Provided
)

Common Configurations

Configuration	Description	Classpath
Compile	Default, production code	Compile + Runtime
Test	Test code only	Test classpath only
Runtime	Runtime dependencies	Runtime classpath
Provided	Provided by runtime environment	Compile only
Optional	Optional dependencies	Optional

Classifiers

Classifiers allow you to request specific variants of artifacts:

build.sbt

libraryDependencies ++= Seq(
  // Main artifact
  "org.apache.spark" %% "spark-core" % "3.5.0",
  
  // Tests artifact with classifier
  "org.apache.spark" %% "spark-core" % "3.5.0" % "test" classifier "tests",
  
  // Sources classifier
  "org.typelevel" %% "cats-core" % "2.10.0" classifier "sources"
)

Common classifiers include sources, javadoc, and tests.

Dependency Exclusions

Exclude transitive dependencies:

Single Exclusion

libraryDependencies += 
  "org.apache.spark" %% "spark-core" % "3.5.0" exclude("org.slf4j", "slf4j-log4j12")

Dependency Overrides

Force specific versions of transitive dependencies:

build.sbt

dependencyOverrides ++= Seq(
  "com.google.guava" % "guava" % "32.1.3-jre",
  "org.scala-lang.modules" %% "scala-xml" % "2.2.0"
)

Use dependency overrides carefully as they can cause compatibility issues if the forced version is incompatible.

Resolvers

Resolvers specify where sbt looks for dependencies:

build.sbt

resolvers ++= Seq(
  "Sonatype OSS Snapshots" at "https://oss.sonatype.org/content/repositories/snapshots",
  "Artima Maven Repository" at "https://repo.artima.com/releases",
  Resolver.mavenLocal
)

Common Resolver Types

Maven Repository

resolvers += "Maven Repo" at "https://repo.example.com/maven2"

Coursier

sbt uses Coursier as the default dependency resolver. Configure Coursier settings:

build.sbt

// Coursier cache directory
csrCacheDirectory := file(sys.props("user.home")) / ".coursier" / "cache"

// Reconciliation strategy for version conflicts
csrReconciliations := Seq(
  "*" -> Reconciliation.Default
)

Conflict Resolution

Handle version conflicts:

1. View evictions

   sbt evicted
   

   Shows which dependency versions were evicted during resolution.

2. Configure conflict manager

   build.sbt

   conflictManager := ConflictManager.strict
   
3. Set eviction error level

   build.sbt

   evictionErrorLevel := Level.Warn
   

Version Schemes

Declare how your project versions are structured:

build.sbt

versionScheme := Some("early-semver")
// Options: "early-semver", "pvp", "semver-spec", "strict"

This helps sbt understand version compatibility.

Intransitive Dependencies

Declare a dependency without its transitive dependencies:

build.sbt

libraryDependencies += 
  "org.apache.kafka" % "kafka-clients" % "3.6.0" intransitive()

Intransitive dependencies may cause runtime errors if required transitive dependencies are missing.

Module Configurations

Configure resolvers per module:

build.sbt

moduleConfigurations += ModuleConfiguration(
  "org.example" %% "*",
  "Custom Resolver" at "https://custom.repo.example.com/"
)

Checking Dependencies

Useful sbt commands for dependency management:

# Show dependency tree
sbt dependencyTree

# Show what dependencies would be added
sbt "show libraryDependencies"

# Update and fetch dependencies
sbt update

# Show evicted dependencies
sbt evicted

# Show classpath
sbt "show Compile/fullClasspath"

Credentials for Private Repositories

Configure credentials for private repositories:

In build.sbt

credentials += Credentials(
  "Sonatype Nexus Repository Manager",
  "nexus.example.com",
  "username",
  "password"
)

Dependency Lock Files

Generate and use dependency lock files for reproducible builds:

# Generate lock file
sbt dependencyLock

# Check lock file is up to date
sbt dependencyLockCheck

Best Practices

1. Use %% for Scala libraries

   Always use %% for libraries compiled against Scala versions to ensure binary compatibility.

2. Scope test dependencies

   Mark test-only dependencies with % Test to keep them out of production classpaths.

3. Monitor evictions

   Regularly run sbt evicted to understand version conflicts in your dependency tree.

4. Pin critical versions

   Use dependencyOverrides for critical dependencies where version mismatches could cause issues.

5. Use version schemes

   Declare your versionScheme to help sbt understand compatibility.