• borosilicate@lemmy.dbzer0.com
    link
    fedilink
    arrow-up
    11
    arrow-down
    1
    ·
    edit-2
    8 hours ago

    Resident (evidently unwelcome) nix evangelist chiming in here.

    The article is about scripts that declare their own dependencies. I wrote another response lamenting the fact that we are talking about dependencies without mentioning IMO the best way to handle dependencies in a script and got flamed by a toxic luddite:

    NIX is the answer.

    Look at what actually gets declared in the article: the library version goes in the file, and the thing that runs the file is whatever brew install hands you that second. Going through OP’s examples:

    Ruby: gem "optimist" has no version at all. Every machine resolves it to whatever is newest on first run.

    Rust: cargo +nightly -Zscript. The +nightly syntax is a rustup proxy feature, so a distro-packaged cargo won’t even parse it. Nightly is a different compiler every day, -Zscript is unstable and its frontmatter syntax has already changed once, clap = "4" and anyhow = "1" are ranges, and there’s no Cargo.lock. Nothing about that script is fixed except its text.

    Haskell: optparse-applicative is pinned exactly, but base >= 4.18 && < 5 accepts any GHC from 9.6 up, and the transitive deps get solved against whatever cabal update fetched. Two people running it a month apart get two build plans. You can add an index-state in a {- project: -} block, which helps with Hackage and does nothing for GHC.

    Python: typer is exact, click and rich and the rest float, and >=3.10 is any interpreter. uv lock --script fixes that by writing a second file, at which point it’s no longer a single-file script.

    Also env -S needs coreutils 8.30+ or a BSD env, and every prerequisite listed is Homebrew on macOS.

    The biggest gap is that none of the examples touches a C library. The first script that needs libpq or openssl or zlib headers is outside what NuGet, Hackage, crates.io or PyPI can describe. You’re back to brew/apt and whatever version happens to be installed.

    Nix has been usable as a shebang interpreter for YEARS, so here is the Haskell example with the compiler, every library, and libc pinned by one commit hash:

    #!/usr/bin/env nix
    #! nix shell --impure --expr ``
    #! nix with (builtins.getFlake ''github:NixOS/nixpkgs/e554fab72f81915600f3f449b786fd9af40439a5'').legacyPackages.${builtins.currentSystem};
    #! nix ghc.withPackages (ps: [ ps.optparse-applicative ])
    #! nix ``
    #! nix --command runghc
    
    import Options.Applicative
    
    data Options = Options
      { name :: String
      }
    
    options :: Parser Options
    options =
      Options
        <$> strArgument
          ( metavar "NAME"
         <> help "Name to greet"
          )
    
    main :: IO ()
    main = do
      opts <-
        execParser $
          info
            (options <**> helper)
            (fullDesc <> progDesc "Say hello")
    
      putStrLn $ "Hello, " <> name opts <> "!"
    

    --impure is only there so it can read the current system. Same GHC and same build of every dependency on any Linux or macOS box, today or in a hundred fifty years. Native deps are the same mechanism: add postgresql to the list.

    When the library isn’t in nixpkgs, nix users pin the artifact by content hash. The article’s babashka example downloads org.babashka/cli from Clojars at runtime. Here the jar is a fixed-output fetch handed to bb on its classpath, so nothing gets resolved when the script runs:

    #!/usr/bin/env nix
    #! nix shell --impure --expr ``
    #! nix with (builtins.getFlake ''github:NixOS/nixpkgs/e554fab72f81915600f3f449b786fd9af40439a5'').legacyPackages.${builtins.currentSystem};
    #! nix let cli = fetchurl { url = ''https://repo.clojars.org/org/babashka/cli/0.12.91/cli-0.12.91.jar''; hash = ''sha256-HPvn4scG4lHZJJLPpXV5ZqbQ17aFZ64AC4fmF5B2HHs=''; };
    #! nix in runCommand ''bb-pinned'' { nativeBuildInputs = [ makeWrapper ]; } ''makeWrapper ${babashka}/bin/bb $out/bin/bb-pinned --set BABASHKA_CLASSPATH ${cli}''
    #! nix ``
    #! nix --command bb-pinned
    
    (require '[babashka.cli :as cli] :reload)
    
    (defn hello [{:keys [name]}]
      (println (str "Hello, " name "!")))
    
    (cli/dispatch
      [{:exec-fn hello
        :args->opts [:name]
        :spec {:name {:positional true :require true :desc "Name to greet"}}}]
      *command-line-args*
      {:prog "hello" :help true})
    

    The :reload matters: bb bundles its own copy of babashka.cli, and without it the require keeps the bundled one. If Clojars ever serves different bytes for that URL, the script refuses to run. This works for a jar with no transitive deps, which this one is. A real dependency tree needs a generated lock file, and at that point you want a flake.nix and flake.lock next to the script, with nix run replacing the shebang.

    What you don’t get for free just by choosing Nix: you need Nix installed (one prerequisite instead of one per language), flakes are still behind an experimental flag upstream, first run is slow, evaluation adds a few hundred ms per run, and the five-line shebang is uglier than anything in the article. And if you write nixpkgs#babashka without a rev you’ve pinned nothing, it follows the registry.

    So I’d say it belongs in the list. It’s the only entry where the interpreter is part of what the script declares.

    • karlhungus@lemmy.ca
      link
      fedilink
      arrow-up
      6
      ·
      8 hours ago

      Thanks! I’ve been nix curious for awhile, but lazyness has won.

      1. Is there some that generates those nix comments for you?

      2. For Dev environments I’m usually interested in the current stable release of whatever language I’m working in, and generally for my tools (nvim, ripgrep, those sorts of things be also on whatever is stable, is there a idiomatic way to get that? And also keep them uotodate?

      Its reasonable to ignore or respond with RTFM

      • borosilicate@lemmy.dbzer0.com
        link
        fedilink
        arrow-up
        4
        ·
        edit-2
        8 hours ago

        No problem. Thanks for being friendly!

        Not RTFM territory at all! ☺️ The manual is honestly one of Nix’s weaker points.

        1. No generator, I wrote those by hand. The multi-line one looks scarier than normal usage though. 95% of the time the whole thing is one line: #! nix shell nixpkgs#ripgrep nixpkgs#jq --command bash. The two ugly bits in my examples both come from a command. The commit hash is nix flake metadata github:NixOS/nixpkgs/nixos-unstable --json | jq -r .locked.rev. The sha256 for the jar I got the lazy way: put a fake hash in, run it, and Nix fails with “specified X, got Y”. Paste Y in. Everybody does it that way and though it can be considered “hacky” you only have to do that dance once when you declare or want to update it to the latest hash.

        For anything bigger than a script you don’t write hashes at all. You write a flake.nix that says “nixpkgs, unstable branch” and Nix generates a flake.lock with the exact commit and hashes, same idea as Cargo.lock or package-lock.json.

        1. Yes, and this is the thing Nix is best at. One naming trap first: for “current stable release of the language” you want the nixos-unstable branch. “Unstable” means the package set rolls forward, not that the packages are betas. It has whatever the latest stable Go/Rust/GHC/etc is, usually within days, and it only advances after the test suite passes. The “stable” branches (nixos-26.05 and so on) freeze versions for six months, which is what you want for a server and usually not for a dev box.

        A dev environment is a flake.nix in the repo root:

        {
          inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-unstable";
        
          outputs = { self, nixpkgs }:
            let
              systems = [ "x86_64-linux" "aarch64-linux" "x86_64-darwin" "aarch64-darwin" ];
              forAllSystems = f: nixpkgs.lib.genAttrs systems (system: f nixpkgs.legacyPackages.${system});
            in
            {
              devShells = forAllSystems (pkgs: {
                default = pkgs.mkShell {
                  packages = [
                    pkgs.go
                    pkgs.gopls
                    pkgs.ripgrep
                  ];
                };
              });
            };
        }
        

        nix develop drops you into a shell with exactly those. First run writes flake.lock, you commit it, and everyone who clones the repo gets identical versions without having to download a bloated Docker image (instead running those dependencies natively in Nix’s sandbox). Updating is nix flake update and then commit the lock. Nothing moves until you run that, so you update when you choose to and if something breaks you git checkout flake.lock and you’re back. If you want it automated there’s a GitHub action (DeterminateSystems/update-flake-lock) that opens a PR with the bumped lock on a schedule, and Renovate handles flake.lock too.

        Add direnv + nix-direnv and the shell loads on cd into the project, so you never type nix develop again. That’s the point where it stops feeling like extra work.

        For personal tools like nvim and ripgrep that you want everywhere and not per project: quick way is nix profile install nixpkgs#neovim nixpkgs#ripgrep and later nix profile upgrade --all. The idiomatic way is home-manager, where your tool list and dotfiles live in one flake in a git repo and a new machine is a clone plus one command. I’d start with the profile commands and a devShell in one project, and only look at home-manager once you’re sure you like it.

        One caveat: for Rust, if you need an exact toolchain version or nightly, nixpkgs only carries current stable, so people use fenix or rust-overlay as a second flake input to declare specific builds in the closure. Most other languages just have versioned attributes like pkgs.python312 or pkgs.jdk21.

  • atzanteol@sh.itjust.works
    link
    fedilink
    English
    arrow-up
    2
    ·
    10 hours ago

    I really wish kscript would catch on. Kotlin is an excellent language and I have enjoyed using it for some utility scripts.

  • AllNewTypeFace@leminal.space
    link
    fedilink
    arrow-up
    1
    ·
    23 hours ago

    How long do most of these take to parse/compile each time you run them? I’m guessing this doesn’t cache compiled artefacts. If the language has a mode to interpret it without compilation (trading runtime speed for compilation overhead) it may make sense, but if not, certainly the Scala and Haskell examples would be agonisingly slow.

    • justcallmelarry@lemmy.dbzer0.com
      link
      fedilink
      arrow-up
      2
      ·
      23 hours ago

      Can’t speak for most of these ecosystems, but uv (for python) does cache the dependencies per version on user level, so it’s def not slower than python is in general, post first run

  • borosilicate@lemmy.dbzer0.com
    link
    fedilink
    arrow-up
    3
    arrow-down
    3
    ·
    edit-2
    9 hours ago

    People around here love to try to do anything in their power to achieve the exact properties of using Nix using ANYTHING other than Nix.

    • TehPers@beehaw.org
      link
      fedilink
      English
      arrow-up
      5
      ·
      10 hours ago

      Mirroring the other comment, but this post really has nothing to do with Nix, so I have no clue where this comment came from.


      But anyway, I think Nix has a few things that turn people away:

      • New language to learn (which is fairly simple, but gets more complicated when you introduce…)
      • Nix flakes? nix-shell? nix-env? Just looking at setting up a shell, you have nix-shell, nix shell, nix develop, and possibly so many others
      • The module system, put lightly, is confusing (for when you need modules)
      • It’s difficult to find which packages are available in nixpkgs (and making a custom derivation is not always trivial)
      • Almost nobody documents how to install their tools with Nix (or add it to a shell.nix or equivalent)
      • … and so on

      Nix is awesome, of course, but it’s far from being the right tool for everyone.

      • borosilicate@lemmy.dbzer0.com
        link
        fedilink
        arrow-up
        1
        arrow-down
        1
        ·
        edit-2
        5 hours ago

        Those are all (IMO completely fair) assessments that can be distilled down to four:

        • nix is a subset of json. If that’s a “hard” language, I don’t know what to tell people. Try GUIX with scheme?
        • many of these points can be distilled to “nix’s documentation is notoriously hard/bad for beginners”
        • most power users use flakes but the community still doesn’t officially accept them (herding cats) even though they are ubiquitous far more sane, standardized way to do many of the things this article spends paragraphs talking about.
        • NixOS search exists so I’m not sure why it’s considered “hard to find packages”

        The title and objective of the article is about managing dependencies of scripts. Author spends many lines talking about “installing” a dependency but that advice falls apart the first time one of those dependencies even slightly changes. This cannot happen with Nix since you’d have to update the lock. That alone is the difference between 100% deterministic to “it doesn’t work on my machine because my package manager serves up X version of this package installed instead of Y in this specific release which has totally different, incompatible commands.” That NEVER happens to me because I have the dependencies LOCKED with nix.

        That is quite literally what nix was built to do; and of course they don’t mention nix which is why I’m here mentioning it. IMO, Nix is incredibly pertinent to exactly this article.

        • TehPers@beehaw.org
          link
          fedilink
          English
          arrow-up
          2
          ·
          56 minutes ago

          nix is a subset of json. If that’s a “hard” language, I don’t know what to tell people. Try GUIX with scheme?

          This ignores how modules and flakes work. You often have to write function expressions, not just pure “JSON”, and it’s often unclear what the function parameters are supposed to be (at least for a beginner).

          NixOS search exists so I’m not sure why it’s considered “hard to find packages”

          The search is great and includes a ton of stuff, but there’s a lot that’s hard to install through it. For example, how do you install an application through Flatpak? How do you install extensions for Firefox or vscodium? How do you install a specific version of an application on an older nixpkgs commit while still installing the current version of other programs through a flake? (I’m guessing through multiple nixpkgs inputs?) How do you install something not in nixpkgs? (A custom derivation, but it’s not easy to build one if you’re a beginner or don’t understand what the program needs during build.)

          These are all things that can be done, but there’s really no specific documentation to help with it.

          What Nix really needs is better documentation and a commitment to (or against) flakes. Once it reaches that point, I think I’d consider it mature enough to recommend. I’ve definitely brought it up at work as an alternative to devcontainers, but it’s a hard sell for a lot of people because it’s so much harder to learn. Personally I use a shared HM config across all my machines, and it’s awesome, but I don’t know anyone else personally who has been able to figure it out.

          • borosilicate@lemmy.dbzer0.com
            link
            fedilink
            arrow-up
            2
            ·
            43 minutes ago

            Well said. I’m glad people are challenging my json characterization. I deserve the flack but perhaps it was just a sensationalist headline to get some discussion going. It worked. ;)

            Thanks for the valuable anecdotal evidence. You do speak the truth. I’ve settled on the fact that the community will never agree to adopt flakes.

            Personally, I have an opinionated stack that is, at this point, self documenting. I maintain a strict style where I add assertions to most things that nix won’t be able to prove conclusively with just a build and cheaper checks that I run before actually building any real derivations. As I said elsewhere, Nix feels like the revolutionary precursor to a sea change in dependency management. But it’s just that: an imperfect technology with enough great ideas for me to adopt it wholesale and use fixing and finding ways around the warts as a way to give back to this open source community.

          • borosilicate@lemmy.dbzer0.com
            link
            fedilink
            arrow-up
            1
            arrow-down
            1
            ·
            6 hours ago

            True!

            I suggest you read everything I write as if I am wearing a slight smirk.

            I can’t help but lament that it (and its cool nerdy brother GUIX, other projects that forked or reimagined Nix, or even programming languages that were built around that “Haskell but hAsH eVeRyThInG” ideal that Unison embodies) will someday be known as the deterministic precursor to whatever effectively supplants the FHS and Docker blight in one fell swoop.

            A machine-independent, deterministic closure that I can spin up in one command is the holy grail, IMO…and it’s pretty achievable even today with the aforementioned tech.

            But, IMO content-addressed and dynamic-derivations bring the nix community to that ideal that Eelco Dolstra envisioned all those years back for now.

    • thingsiplay@lemmy.ml
      link
      fedilink
      arrow-up
      4
      arrow-down
      2
      ·
      17 hours ago

      People around here love to do anything in their power to achieve the exact properties of using Nix using ANYTHING other than Nix.

      And I don’t think its a bad thing. Otherwise we become too dependent on one platform / solution / developers. I would also like to have some Nix superpowers, without relying on Nix. However I don’t know what this post has anything to do with Nix.

      • borosilicate@lemmy.dbzer0.com
        link
        fedilink
        arrow-up
        2
        arrow-down
        1
        ·
        12 hours ago

        Good point. There’s zero danger of that though. The overused dependency in most everyone’s stack is actually Docker. If anything, Nix is a better way to lock/archive state for future runs.

        • thingsiplay@lemmy.ml
          link
          fedilink
          arrow-up
          2
          arrow-down
          1
          ·
          11 hours ago

          Nix might be the better concept overall, but its a specific concept that is not available everywhere. Docker on the other hand can be installed and used on any distribution, and removed too. Once you rely on Nix and its system, you can’t just remove it, its your lifestyle now.

          • borosilicate@lemmy.dbzer0.com
            link
            fedilink
            arrow-up
            2
            arrow-down
            1
            ·
            edit-2
            11 hours ago

            That’s silly. There’s a lot you can say about nix but you clearly don’t know much about it if you think it can’t be used across many platforms.
            You sure about that? https://github.com/nix-community/nixos-anywhere

            https://dev.to/jajera/using-nix-on-windows-the-right-way-14ki

            Also, you can’t remove Nix? Are you sure? You seem to not know anything at all about Nix but are just parroting various untrue things about it.

            • thingsiplay@lemmy.ml
              link
              fedilink
              arrow-up
              1
              arrow-down
              2
              ·
              11 hours ago

              We are not talking about the package manager.

              Also, you can’t remove Nix?

              I don’t think you understood what I said.

              • borosilicate@lemmy.dbzer0.com
                link
                fedilink
                arrow-up
                1
                arrow-down
                1
                ·
                11 hours ago

                We literally are talking about the package manager.

                I understood very well. Maybe you don’t understand what you said.

                Great job gaslighting me:

                Once you rely on Nix and its system, you can’t just remove it, its your lifestyle now.

                • thingsiplay@lemmy.ml
                  link
                  fedilink
                  arrow-up
                  1
                  arrow-down
                  1
                  ·
                  11 hours ago

                  Dude whats your problem now? Take a deep breath.

                  You understand that saying something does not always mean “literal”. Who the fuck think that when I say you cannot remove it, that I would mean it literally? I said its becoming part of your lifestyle. Think about it and stop being a little child with the replies. Nobody is gaslighting you, in fact you are gaslighting me or you are incredible stupid at understanding other people.

                  So with those replies of you its clear that you are toxic and this is what you get for. Have a nice day.