7.3 KiB
7.3 KiB
% Two decades of Open Source % Julian Ospald % Sep 20, 2024
Introduction
About me
- From Germany
- Studied CS
- Haskell developer
- I love open source
Professional career
- Software Engineer in R&D (automotive industry)
- Go Backend Developer (online advertisement platform
- Haskell Developer at Capital Match (invoice financing platform in Singapore)
- Haskell Developer at IOHK (Cardano Blockchain)
- Haskell Freelancer (blockchain and others)
- Haskell Developer at Standard Chartered Bank
- Haskell Freelancer (chimney sweeper app for german businesses)
Open Source career
- Gentoo Linux developer (core team), 2012-2016
- Ebuild development (packaging)
- Code review
- Development of a git workflow
- Author of GHCup (the Haskell installer), ca. 2019
- Maintainer of Haskell core libraries: filepath, unix, os-string, file-io
- Implementation of the Abstract FilePath Proposal
- Member of the Haskell Core Libraries Comittee 2023-2026
- Haskell Influencer (Haskell Foundation, ...)
Open Source
What is Open Source
- {#id .class height=32px} A group of licenses (see OSI)
- Not free software
- Not copyleft
- 🧑🤝🧑 A community
- volunteers
- companies
- 🔮 A philosophy
- sharing
- collaboration
- transparency
Popular Open Source projects
- {#id .class height=32px} Linux kernel
- 1500 developers from 200-250 companies
- {#id .class height=32px} Firefox
- {#id .class height=32px} VSCode
- {#id .class height=32px} Blender
- {#id .class height=32px} GHC (The Haskell compiler)
Value proposition of Open Source
- ⚗️ the scientific method
- share your results
- allow people to replicate it
- 🔓 access to a community
- users
- collaborators
- 🕸️ network effects
Reality of Open Source
- most projects...
- are one-man shows
- have no users
- are underdocumened
- have horrible code
- writing new code is easy, maintenance is hard
- most maintainers
- don't get paid
- will stop maintenance at some point
- don't care much about their users
Gentoo and package management
What is Gentoo
- a Linux distribution
- rolling release
- source based
- 19000 packages (program, library, ...)
- 200 core developers (at its peak)
- over 1000 contributors
How does a Linux distro work (relationships)
How does a Linux distro work (activities)
A typical ebuild
EAPI=8
DESCRIPTION="A dummy package"
HOMEPAGE="https://dummy.org"
SRC_URI="https://github.com/dummy/dummy/archive/refs/tags/${PV}.tar.gz -> ${P}.tar.gz"
LICENSE="BSD-3"
SLOT="0"
KEYWORDS="~amd64 ~x86 "
IUSE="debug"
RDEPEND="dev-util/boost"
PATCHES=( "${FILESDIR}"/${PN}-4.9.2-disable_python_rpath.patch)
src_configure() {
econf $(use_enable debug)
}
src_compile() {
emake
}
src_test() {
emake test
}
src_install() {
emake DESTDIR="${D}" install
}
Packaging challenges
- no standard on build systems (make, autotools, meson, cmake, ...)
- => an abstraction over build systems
- thousands of different execution environments (fragility)
- system configuration
- package configuration
- platform, architecture
- reverse dependencies
- shipping a "chain" instead of a single artifact
- high impact on small mistakes (e.g. assuming a specific shell)
Packaging challenges (pt 2.)
- communication between teams/maintainers
- execution of large changes
- e.g. introduction of LibreSSL
- e.g. changing of fundamental workflows (from CVS to git)
- monitoring upstream changes and making decisions about compatibility/stability
- when to update
What is a Distro really?
- a user experience
- LTS distros vs rolling release
- binary vs source based
- choice of init system
- plug and play (everything works)
- deviating from the happy path (fixing issues)
- combining components to a coherent system (init system, coreutils, kernel, ...)
- a choice of defaults
Why Gentoo made me a better programmer
- primary packaging skill: being meticulous
- small mistakes -> big impact
- long term maintenance of small code pieces
- intense review culture
- strict policies and workflow guidelines
- how to learn complex system
GHCup
Demo
State of 2019 (Haskell Installers)
- stack is the only "Haskell Installer"
- no unified alternative for cabal users
- distro packages, nix, manual installs, ...
- 😭
How it started
- 🤹 small team at work (Capital Match), using different platforms
- originally used stack
- distro packages constantly out of date
- 🦾 first version was 165 LOC
- Posix shell
- {#id .class height=32px} only supported linux and mac
- {#id .class height=32px} inspired by rustup
- support from haskell.org
GHCup today
- over 17k LOC Haskell
- supports all platforms: Linux, Windows, macOS, FreeBSD
- first thing new Haskell users get exposed to
What is GHCup (simplified)?
curl -s -L \
'https://downloads.haskell.org/~ghc/9.6.5/ghc-9.6.5-x86_64-fedora33-linux.tar.xz' |
tar -xJ -C /tmp &&
cd /tmp/ghc-9.6.5-x86_64-unknown-linux/ &&
./configure --prefix="$HOME/.local" &&
make install &&
rm -rf /tmp/ghc-9.6.5-x86_64-unknown-linux/
What is GHCup really?
- {#id .class width=32 height=32px} installer (portable)
- {#id .class width=32 height=32px} distribution channel
- {#id .class width=32 height=32px} feedback channel
- {#id .class width=32 height=32px} testing/QA gateway
- {#id .class width=32 height=32px} provider of sane defaults (e.g. "recommended" GHC version)
- {#id .class width=32 height=32px} glue for holistic toolchain experience
- VSCode, stack, cabal-install integration
- {#id .class width=32 height=32px} CI provisioning (e.g. github actions)
The difference to Gentoo
- one-man project (mostly)
- much tighter coupling between upstream (e.g. GHC developers) and downstream (GHCup developers)
- heavier on relationship issues
- much more responsibility
- position of authority
Relationships in detail
Dependencies:
- supported tools
- GHC
- Cabal
- HLS
- Stack
- decisions that affect us
- release frequency
- upstream CI
- platform support
- binary distributions (the
.tar.gz
/.zip
)
Relationships in detail
Dependents: