@filippo Interesting stuff

One nitpick, or perhaps more likely a misunderstanding on my part: I wasn't sure what you meant by lockfiles applying recursively, but if that's referring to pinning the indirect dependencies of the main package, then at least in Python, they *do* apply recursively - at least as far as I understand it, a lockfile is supposed to specify exact versions of every dependency all the way down the graph. The idea is that if you install the exact packages listed in the lockfile and no others in an empty environment, everything should work. Or did you mean something else by applying recursively?

@michael awesome, that’s exactly what I wrote it for!

You might want to reconsider using whatever generated that notification: if they get something so basic wrong, it’s unlikely they’ll be doing everything else right!

@diazona they don’t apply to dependents. Click on the linked post by Russ Cox for a full explanation.

@filippo

when Russ Cox started publishing his research on packaging and minimal version selection, i was deep into Python packaging, writing my own package manager (DepHell), dependency resolution included. and i was honestly impressed. then we actually got it in Go and practice confirmed the theory: this is THE BEST package manager and dependency resolver ever made. if nothing else, i'd use Go just for that.

@risottobias expanded a bit the part that says what it's for!

@filippo but there's no hashes in gomod, is there?

@raito correct, go.mod has the versions, go.sum is a dumb mapping of versions to hashes.

@filippo so I feel like you almost never need to look at go.sum except if you do use a system that allows only fixed output fetches and then go.sum is useful to predict more or less that hash, right?

@filippo Yes I read that post; that's what helped me understand enough to formulate my question. It's about npm, and maybe npm does lockfiles differently, I wouldn't know, but what he's saying there does not accurately describe how any of the Python installers I'm familiar with handle lockfiles.

@raito yeah, cmd/go/internal/modfetch needs the hashes when downloading contents. But essentially no one reimplements that part.

@diazona if you add foo to xxx’s dependencies, and foo depends on bar, which version of bar is used?

@filippo Is this about lockfiles? If the installation is being done from a lockfile, then it's whichever version of bar is specified in the lockfile. Otherwise, depends on the resolver and how it's configured, but probably the latest available version of bar, if there's no further constraint on its version.

I'm not sure I see where you're going with this.

@filippo Well, the Nix ecosystem definitely had appetite for that but the usage of dirhash made that quite hard because AFAIK it's not easy to go from dirhash to some raw SHA-256 or similar.

@diazona what does “being done from a lockfile” mean in this context?

You are in xxx. You add foo. Which version of bar do you get? The latest or the one in foo’s lockfile?

In Go, you get the one in foo’s go.mod. Which is why I say go.mod applies to dependents like manifests and unlike lockfiles, despite having lockfile-like precision.

@raito yeah it’s a trade-off, dirhash OTOH avoids the need to stabilize the archive format, compression, and metadata including timestamps

@filippo yep, it's fairly understandable why dirhash was chosen; unfortunately, Go (among application ecosystems) is quite unique in having solved this problem. Someday, Nix will support extensible fixed-output primitives so that dirhash can be explained to Nix and then Go modules can be natively downloaded by Nix and fed to the rest of the Go packaging machinery.

@raito @filippo isn't NAR basically defunctionalization of dirhash?

@arianvp @filippo i think NAR is more stupid than dirhash

@filippo

"what does 'being done from a lockfile' mean in this context?": You have a program which is capable of installing packages. How do you tell it which packages to install? When I say "being done from a lockfile", I mean that the way you tell the program which packages to install is by giving it a lockfile. (As opposed to, say, giving it the name of a package.)

"You are in xxx. You add foo. Which version of bar do you get? The latest or the one in foo’s lockfile?": What does it mean to be "in" xxx? Based on previous messages, I thought xxx was a package?

P.S. not trying to be difficult here, I'm just genuinely confused

@diazona @filippo
I don't understand why such a thing would be called a lockfile. I've only ever heard "lockfile" used to mean a file that is temporarily created to indicate that some resource should be considered "locked", and unavailable for use by e.g. another process running the same or a related program. In other words, using the filesystem to implement a mutex. (This method only works for programs that agree to the usage convention; the OS kernel doesn't enforce anything.)