UV Sync Vs UV Pip Install Understanding Differing Behaviors With Extras

Hey guys! Let's dive into a quirky issue I stumbled upon while working with uv, a tool that's supposed to make Python dependency management smoother. I wanted to share my findings on the differing behaviors between uv sync and uv pip install when dealing with extras and path dependencies. Trust me, this can be a real head-scratcher if you're not expecting it.

The Mystery of the Missing Modules

So, here’s the deal. I was wrestling with a setup where a path dependency could live in two different spots: ../../common_modules for my local setup and ./common_modules in our pipeline environment. I figured, no sweat, I’ll just use uv to handle this. I set up my pyproject.toml like this:

[project]
name = "teststep"
version = "0.1.0"
description = ""
authors = [{ name = "lilian-delouvy" }]
requires-python = ">=3.11, <3.12"
dependencies = []

[project.optional-dependencies]
local = ["common-modules"]
pipeline = ["common-modules"]

[tool.uv.sources]
common-modules = [
    { path = "../../common_modules", editable = true, extra = "local" },
    { path = "./common_modules", extra = "pipeline" }
]

[tool.uv]
package = true
conflicts = [
    [
        { extra = "local" },
        { extra = "pipeline" }
    ]
]

[tool.hatch.build.targets.sdist]
include = ["src/teststep"]

[tool.hatch.build.targets.wheel]
include = ["src/teststep"]

[tool.hatch.build.targets.wheel.sources]
"src/teststep" = "teststep"

[tool.hatch.metadata]
allow-direct-references = true

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

Diving Deep into the pyproject.toml Configuration

Let's break down this pyproject.toml configuration, guys. At the heart of this setup is the [tool.uv.sources] section. This is where the magic (or in this case, the mystery) happens. We're telling uv that common-modules can be found in two different locations, depending on the environment. Locally, it's chilling at ../../common_modules, and in the pipeline, it's hanging out at ./common_modules. The extra field is key here, as it allows us to specify which environment each path belongs to.

Now, the [tool.uv] section is where we set some general uv configurations. package = true tells uv to treat path dependencies as regular packages, which is crucial for our setup. The conflicts section is a neat way to ensure that we don't accidentally try to install both the local and pipeline versions of common-modules at the same time. This is super important for maintaining a clean and predictable environment.

The [project.optional-dependencies] section defines the local and pipeline extras. These extras are essentially labels that we can use to activate specific sets of dependencies. In our case, we're saying that the local extra should include the common-modules dependency when we're working locally, and the pipeline extra should include it when we're in the pipeline environment.

The rest of the pyproject.toml file deals with build configurations for Hatch, which is our build backend. We're specifying which files to include in the source distribution (sdist) and wheel packages, and we're also setting up metadata for Hatch. The [tool.hatch.metadata] section with allow-direct-references = true is particularly important because it allows us to reference path dependencies directly, which is what we're doing with common-modules.

So, with this setup, we're aiming for a flexible system where we can easily switch between local and pipeline environments without having to manually juggle dependencies. The goal is to use uv to handle the complexities of path resolution and extra selection, making our development workflow smoother and more efficient. But, as we'll see, things didn't quite go as planned.

The Unexpected Error with uv sync

I ran uv sync --extra local, fully expecting it to grab the common_modules from ../../common_modules. But, bam! I got this error:

error: Distribution not found at: file:///C:/<MY_PATH>/teststep/common_modules

What the heck? It seemed like uv was still trying to resolve the “pipeline” path, even though I specifically asked for the “local” extra. This was not the behavior I was expecting, and it threw a wrench in my plans.

The Curious Case of uv pip install

Now, here's where it gets even more interesting. I decided to try uv pip install -r pyproject.toml --extra local. To my surprise, this worked perfectly! It installed the common_modules from ../../common_modules without a hitch. What gives?

The Plot Thickens: Differing Behaviors

This is where I realized there’s a significant difference in how uv sync and uv pip install handle extras and path dependencies. From my understanding, these two commands should behave the same in this scenario. I mean, uv sync --extra local and uv pip install -r pyproject.toml --extra local should, in theory, achieve the same outcome: install the dependencies specified in the pyproject.toml file, considering the “local” extra. But clearly, they don’t.

Unraveling the Mystery: Why the Discrepancy?

So, what’s the deal? Why is uv sync behaving differently from uv pip install? It seems like uv sync might be trying to resolve all possible paths, regardless of the active extras, while uv pip install is more selective and respects the extras. This is a crucial distinction, and it can lead to some unexpected behavior if you're not aware of it.

Potential Causes and Considerations

There are a few potential reasons for this discrepancy. One possibility is that uv sync is designed to be more comprehensive in its dependency resolution, attempting to identify all potential dependencies upfront. This could be beneficial in some scenarios, but in our case, it leads to an error because it tries to resolve the “pipeline” path even when we only want the “local” one.

Another possibility is that there’s a bug or an unintended behavior in uv sync. Given that uv is still a relatively young project, it’s not uncommon to encounter these kinds of quirks. The developers are actively working on improving the tool, so it’s possible that this issue will be addressed in a future release.

It’s also worth considering the complexity of our setup. We’re dealing with path dependencies, extras, and conditional paths, which is a fairly advanced use case. It’s possible that uv is struggling to handle all these factors in the way we expect. This highlights the importance of thoroughly testing your dependency management setup, especially when using more complex configurations.

The Impact on Development Workflows

The differing behaviors between uv sync and uv pip install can have a significant impact on development workflows. If you’re relying on uv sync to manage your dependencies, you might encounter unexpected errors and have to resort to uv pip install as a workaround. This can be frustrating and time-consuming, especially if you’re not aware of the issue.

It also raises questions about the intended use cases for uv sync and uv pip install. If they’re supposed to be interchangeable, then their differing behaviors are a problem. If they’re designed for different purposes, then it’s important to clearly document these differences so that users can choose the right tool for the job.

Platform and Version Details

Just to give you the full picture, I encountered this issue on Windows locally and on Ubuntu in our pipeline environment. I was using uv version 0.7.15 (4ed9c5791 2025-06-25) and Python 3.11. Knowing the platform and version details can be crucial for debugging and reproducing issues, so I wanted to make sure to include them.

Wrapping Up: A Call for Clarity and Consistency

So, there you have it. A tale of two uv commands and their slightly different personalities. It’s a reminder that even the coolest tools can have their quirks, and it’s important to understand how they work under the hood.

The Importance of Clear Documentation and Communication

This whole experience underscores the importance of clear documentation and communication within the uv community. If there are intended differences between uv sync and uv pip install, it’s crucial to document these differences clearly so that users can make informed decisions about which command to use. Similarly, if the differing behavior is a bug, it’s important to communicate this to the community and provide updates on the progress of the fix.

In the meantime, I’ll probably stick to using uv pip install when dealing with extras and path dependencies. It seems to be the more reliable option in this scenario. But I’m definitely keeping an eye on uv and its development. It’s a promising tool, and I’m excited to see how it evolves.

Final Thoughts and Recommendations

In conclusion, the differing behaviors between uv sync and uv pip install when dealing with extras and path dependencies can be a real gotcha. It’s essential to be aware of this issue and to test your dependency management setup thoroughly, especially when using more complex configurations. If you encounter similar issues, consider using uv pip install as a workaround and keep an eye on the uv project for updates.

And hey, if you’ve experienced something similar, I’d love to hear about it in the comments! Let’s learn from each other and make our Python dependency management a little less mysterious.