diff --git a/QR.png b/QR.png new file mode 100644 index 0000000..7b598de Binary files /dev/null and b/QR.png differ diff --git a/custom.css b/custom.css new file mode 100644 index 0000000..17ca1f1 --- /dev/null +++ b/custom.css @@ -0,0 +1,24 @@ +h1 { + text-align: center; +} + +div > p { + padding-left: 40px; +} + +div > ul { + padding-left: 53px; +} + +.centered { + p { + padding-left: 0px !important; + margin-left: 0px !important; + margin-right: 60px; + padding-right: 20px; + } + ul { + padding-left: 0px !important; + } + text-align: center; +} diff --git a/jiddu.jpg b/jiddu.jpg new file mode 100644 index 0000000..fbf9192 Binary files /dev/null and b/jiddu.jpg differ diff --git a/notes.md b/notes.md index 35cb01d..9dcafba 100644 --- a/notes.md +++ b/notes.md @@ -10,45 +10,62 @@ - functions - libraries - programs (unix) + - lesson: specifications - LSP (open source milestone) + - lesson: caring about features and code instead of maintenance and collaborations -- lesson: dicatorships work +- lesson: users (user experience) + - what if you diverge from the happy path -- decision making (processes) - - lightweight when risk of mistakes is low (can revert?) +- lesson: how to drive change + - good value proposation (e.g. when breaking backwards compat) + - risk: can you revert? -- tests in CI are garbage + +- you guessed it, I like small programs + +- self-perception/myth + +- how to test (on the end users system) + - tests in CI are garbage + - tests vs good code - reverse dependencies <-> me <-> users - collaboration vs boundaries, communication -- what distribution work taught me for programming +- autotools intro? + + +- mindful about what you don't know + + - posix principles and their connection to functional programming (streams) - navigation - strings -- open source politics -- how to drive change -- how to handle contributions (contribution experience, PRs, documentation, mentoring,. ..) - collaboration + - open source politics + - how to drive change + - how to handle contributions (contribution experience, PRs, documentation, mentoring,. ..) + - respect other projects when contributing + - bus factor + - enabling and supporting (switching from coding wizard to support role) + - boundaries vs collaboration + - relationship between industry and FOSS -- what is the main currency (money vs energy) -- bus factor + - feedback from universities regarding Haskell tooling -- respect other projects when contributing -- enabling and supporting (switching from coding wizard to support role) + - project life cycles - support -- stability vs. .. -- boundaries vs collaboration - trust, respect, relationship - working mode in open source - dealing with expectations -- how to test (on the end users system) -- what if you diverge from the happy path + +- stability vs. .. - why is stability an interesting goal? diff --git a/nus.html b/nus.html index b307be5..e4a746f 100644 --- a/nus.html +++ b/nus.html @@ -1,6 +1,5 @@ - - + + @@ -9,90 +8,88 @@ Two decades of Open Source - - +code{white-space: pre-wrap;} +span.smallcaps{font-variant: small-caps;} +div.columns{display: flex; gap: min(4vw, 1.5em);} +div.column{flex: auto; overflow-x: auto;} +div.hanging-indent{margin-left: 1.5em; text-indent: -1.5em;} + +ul.task-list[class]{list-style: none;} +ul.task-list li input[type="checkbox"] { +font-size: inherit; +width: 0.8em; +margin: 0 0.8em 0.2em -1.6em; +vertical-align: middle; +} +.display.math{display: block; text-align: center; margin: 0.5rem auto;} + +pre > code.sourceCode { white-space: pre; position: relative; } +pre > code.sourceCode > span { line-height: 1.25; } +pre > code.sourceCode > span:empty { height: 1.2em; } +.sourceCode { overflow: visible; } +code.sourceCode > span { color: inherit; text-decoration: inherit; } +div.sourceCode { margin: 1em 0; } +pre.sourceCode { margin: 0; } +@media screen { +div.sourceCode { overflow: auto; } +} +@media print { +pre > code.sourceCode { white-space: pre-wrap; } +pre > code.sourceCode > span { display: inline-block; text-indent: -5em; padding-left: 5em; } +} +pre.numberSource code +{ counter-reset: source-line 0; } +pre.numberSource code > span +{ position: relative; left: -4em; counter-increment: source-line; } +pre.numberSource code > span > a:first-child::before +{ content: counter(source-line); +position: relative; left: -1em; text-align: right; vertical-align: baseline; +border: none; display: inline-block; +-webkit-touch-callout: none; -webkit-user-select: none; +-khtml-user-select: none; -moz-user-select: none; +-ms-user-select: none; user-select: none; +padding: 0 4px; width: 4em; +color: #aaaaaa; +} +pre.numberSource { margin-left: 3em; border-left: 1px solid #aaaaaa; padding-left: 4px; } +div.sourceCode +{ } +@media screen { +pre > code.sourceCode > span > a:first-child::before { text-decoration: underline; } +} +code span.al { color: #ff0000; font-weight: bold; } +code span.an { color: #60a0b0; font-weight: bold; font-style: italic; } +code span.at { color: #7d9029; } +code span.bn { color: #40a070; } +code span.bu { color: #008000; } +code span.cf { color: #007020; font-weight: bold; } +code span.ch { color: #4070a0; } +code span.cn { color: #880000; } +code span.co { color: #60a0b0; font-style: italic; } +code span.cv { color: #60a0b0; font-weight: bold; font-style: italic; } +code span.do { color: #ba2121; font-style: italic; } +code span.dt { color: #902000; } +code span.dv { color: #40a070; } +code span.er { color: #ff0000; font-weight: bold; } +code span.ex { } +code span.fl { color: #40a070; } +code span.fu { color: #06287e; } +code span.im { color: #008000; font-weight: bold; } +code span.in { color: #60a0b0; font-weight: bold; font-style: italic; } +code span.kw { color: #007020; font-weight: bold; } +code span.op { color: #666666; } +code span.ot { color: #007020; } +code span.pp { color: #bc7a00; } +code span.sc { color: #4070a0; } +code span.ss { color: #bb6688; } +code span.st { color: #4070a0; } +code span.va { color: #19177c; } +code span.vs { color: #4070a0; } +code span.wa { color: #60a0b0; font-weight: bold; font-style: italic; } + + + +
@@ -102,6 +99,21 @@ Julian Ospald

Sep 20, 2024

+
+

Follow the presentation

+

+
+
+

Structure of this talk

+
    +
  1. Introduction (about me and my career)
  2. +
  3. Open Source (what it is and its value)
  4. +
  5. First chapter: Gentoo and package management
  6. +
  7. Second chapter: GHCup
  8. +
  9. Third chapter: Haskell Core Libraries
  10. +
  11. Lessons learned
  12. +
+

Introduction

@@ -135,14 +147,12 @@ financing platform in Singapore)
  • Author of GHCup (the Haskell installer), ca. 2019
  • Maintainer of Haskell core libraries: filepath, unix, os-string, file-io
  • -
  • Implementation of the Abstract +
  • Implementation of the Abstract FilePath Proposal
  • Member of the Haskell Core Libraries Comittee 2023-2026
  • Haskell Influencer (Haskell Foundation, …)
  • @@ -156,7 +166,7 @@ FilePath Proposal

    What is Open Source

    @@ -229,9 +243,8 @@ Blender
    -
    -

    Gentoo and package management

    +
    +

    First chapter: Gentoo and package management

    @@ -247,21 +260,17 @@ class="title-slide slide section level1">
  • over 1000 contributors
  • -
    +

    How does a Linux distro work (relationships)

    -

    +

    -
    +

    How does a Linux distro work (activities)

    -

    +

    A typical ebuild

    -
    EAPI=8
    +
    EAPI=8
     
     DESCRIPTION="A dummy package"
     HOMEPAGE="https://dummy.org"
    @@ -312,8 +321,8 @@ class="sourceCode bash">
    -

    Packaging challenges (pt 2.)

    +
    +

    Packaging challenges (pt. 2)

    • communication between teams/maintainers
    • execution of large changes @@ -344,14 +353,28 @@ kernel, …)
    • a choice of defaults
    +
    +

    Programming lessons

    +
      +
    • primary packaging skill: being meticulous +
        +
      • small mistakes -> big impact
      • +
      • being as precise as possible about what you want to achieve
      • +
    • +
    • long term maintenance of small code pieces
    • +
    • intense review culture
    • +
    • strict policies and workflow guidelines
    • +
    • how to learn complex system
    • +
    +
    -
    -

    GHCup

    +
    +

    Second chapter: GHCup

    -
    -

    Demo

    -

    +
    +

    Demo

    +

    State of 2019 (Haskell Installers)

    @@ -374,19 +397,18 @@ kernel, …)
    • Posix shell
    -
  • only +
  • only supported linux and mac
  • -
  • inspired by +
  • inspired by rustup
  • support from haskell.org
  • GHCup today

    -

    Haskell +

    Haskell Survey 2022:

    -

    +

    • over 17k LOC Haskell
    • supports all platforms: Linux, Windows, macOS, FreeBSD
    • @@ -406,36 +428,18 @@ Survey 2022:

      What is GHCup really?

        -
      • installer (portable)
      • -
      • distribution channel
      • -
      • feedback channel
      • -
      • +
      • installer (portable)
      • +
      • distribution channel
      • +
      • feedback channel
      • +
      • testing/QA gateway
      • -
      • +
      • provider of sane defaults (e.g. “recommended” GHC version)
      • -
      • glue for holistic toolchain experience +
      • glue for holistic toolchain experience
        • VSCode, stack, cabal-install integration
      • -
      • CI provisioning (e.g. github actions)
      • -
      -
      -
      -

      The difference to Gentoo

      -
        -
      • one-man project (mostly)
      • -
      • much tighter coupling between upstream (e.g. GHC developers) and -downstream (GHCup developers) -
          -
        • heavier on relationship issues
        • -
      • -
      • much more responsibility
      • -
      • position of authority
      • +
      • CI provisioning (e.g. github actions)
      @@ -459,19 +463,17 @@ downstream (GHCup developers)
    -
    -

    Relationships in detail

    +
    +

    Relationships in detail (pt. 2)

    Dependents:

      -
    • +
    • Haskell developers
      • beginners, advanced, students, companies
    • -
    • end users (e.g. compiling pandoc from source)
    • -
    • GitHub CI +
    • end users (e.g. compiling pandoc from source)
    • +
    • GitHub CI
      • GitHub images, Haskell repos
    • @@ -481,73 +483,267 @@ height="32" /> GitHub CI
  • 🧰 tools
  • -
    -

    Other

    +
    +

    Programming lessons

      -
    • the idea of a distribution is to create a user experience

      +
    • writing a small single-purpose program from scratch
    • +
    • how to design command line interfaces
    • +
    • high impact of decisions (not just mistakes)
        -
      • you log into your computer and install a program and everything just -works
      • -
      • or: something doesn’t work… what are your next steps?
      • +
      • bugs now affect GitHub CI and companies
      • +
      • can make “Haskell” look bad
    • -
    • you create an experience

    • -
    • the distribution is that brings all the pieces together: -installation, service management (systemd, openrc, initd), kernel -updates, support

    • -
    • lesson: composition

      +
    • no one to review
        -
      • functions
      • -
      • libraries
      • -
      • programs (unix)
      • +
      • => review your own code
    • -
    • lesson: specifications

      +
    • constantly thinking about ways to improve reliability
        -
      • LSP (open source milestone)
      • +
      • can’t rely on anyone else to catch bugs
    • -
    • lesson: caring about features and code instead of maintenance and -collaborations

    • -
    • dicatorships work

    • -
    • decision making (processes)

      -
        -
      • lightweight when risk of mistakes is low (can revert?)
      • -
    • -
    • tests in CI are garbage

    • -
    • reverse dependencies <-> me <-> users

    • -
    • collaboration vs boundaries, communication

    • -
    • what distribution work taught me for programming

    • -
    • posix principles and their connection to functional programming -(streams)

    • -
    • strings

    • -
    • open source politics

    • -
    • how to drive change

    • -
    • how to handle contributions (contribution experience, PRs, -documentation, mentoring,. ..)

    • -
    • collaboration

    • -
    • relationship between industry and FOSS

    • -
    • what is the main currency (money vs energy)

    • -
    • bus factor

    • -
    • feedback from universities regarding Haskell tooling

    • -
    • respect other projects when contributing

    • -
    • enabling and supporting (switching from coding wizard to support -role)

    • -
    • project life cycles

    • -
    • support

    • -
    • stability vs. ..

    • -
    • boundaries vs collaboration

    • -
    • trust, respect, relationship

    • -
    • working mode in open source

    • -
    • dealing with expectations

    • -
    • how to test (on the end users system)

    • -
    • what if you diverge from the happy path

    • -
    • why is stability an interesting goal?

    +
    +

    The difference to Gentoo

    +

    Both deal with installation, but…

    +
      +
    • more code to maintain (not just packaging) for me
    • +
    • one-man project (mostly)
    • +
    • much tighter coupling between upstream (e.g. GHC developers) and +downstream (GHCup developers) +
        +
      • heavier on relationship issues
      • +
    • +
    • less dependencies, but much more responsibility
    • +
    • position of authority +
        +
      • what to consider?
      • +
    • +
    • most of my work today is support
    • +
    +
    + +
    +

    Third Chapter: Haskell Core Libraries

    + +
    +
    +

    What are Haskell Core Libraries?

    +
      +
    • bundled with the compiler
    • +
    • fundamental building blocks (primitives)
    • +
    • base library +
        +
      • available to all programs by default
      • +
      • contains the “Prelude” (standard library)
      • +
    • +
    +
    +
    +

    Core libraries I maintain

    +
      +
    • filepath
    • +
    • unix
    • +
    • os-string
    • +
    • file-io
    • +
    +
    +
    +

    Challenges

    +
      +
    • changes are extremly expensive
    • +
    • writing good primitives is hard (non-specific APIs)
    • +
    • lots of odd knowledge +
        +
      • e.g. Windows filepaths +
          +
        • C:foo
        • +
        • /bar
        • +
        • \\?\GLOBALROOT\Device\Harddisk0\Partition2\foo\bar
        • +
      • +
      • Posix standard
      • +
    • +
    • portability
    • +
    +
    +
    +

    Core Libraries Committee

    +
      +
    • 7 members
    • +
    • manages API changes of base only +
        +
      • requires a proposal
      • +
      • requires impact assessment for breaking changes
      • +
      • requires an up-front implementation of the change
      • +
    • +
    • ensures other core libraries have active maintainers +
        +
      • does not interfere with maintenance
      • +
    • +
    +
    +
    +

    Driving changes across core libraries (case study)

    +

    Abstract FilePath Proposal:

    +
      +
    • Haskell String type: type String = [Char] +
        +
      • Char is a unicode code point
      • +
      • not bytes
      • +
      • is interpreted (decoded)
      • +
      • depends on locale
      • +
    • +
    • affects most core libraries
    • +
    • implement as a breaking change (base), or… +
        +
      • in “user-space”
      • +
    • +
    • lack of higher authority +
        +
      • building consensus
      • +
      • convincing multiple maintainers
      • +
      • patching many libraries
      • +
    • +
    • open source politics
    • +
    +
    +
    +

    Programming lessons

    +
      +
    • how to design good primitives +
        +
      • as opposed to abstractions
      • +
    • +
    • considering every impact of API changes
    • +
    • doing history research on past design choices +
        +
      • important design decisions may not be documented
      • +
      • may look innocent
      • +
      • chaning them might be devastating
      • +
    • +
    +
    + +
    +

    Lessons Learned

    + +
    +
    +

    Collaboration

    +
    +
      +
    • main currency in Open Source is energy
    • +
    • treat contributors like kings
    • +
    • be mindful about boundaries (tricky balance)
    • +
    • respect other projects workflows
    • +
    • driving large changes requires +
        +
      • consensus
      • +
      • support
      • +
      • a good value proposition
      • +
      • a good execution plan (risk, breakage, …)
      • +
    • +
    • Haskell Foundation
    • +
    +
    +
    +
    +

    Project maintenance

    +
    +
      +
    • dicatorships work, but are not sustainable
    • +
    • plan for your departure
    • +
    • bus factor is your constant enemy
    • +
    • good decision making processes +
        +
      • lightweight when risk is low
      • +
      • elaborate when risk is high
      • +
    • +
    • actively think about the contribution experience +
        +
      • comment early
      • +
    • +
    • how to maintain the project vision?
    • +
    +
    +
    +
    +

    User Experience

    +
    +
      +
    • UX is harder than CS +
        +
      • yet often an afterthought
      • +
    • +
    • toolchains often lack a holistic UX vision
    • +
    • UX vision gets easily lost in “maintenance mode” +
        +
      • feature creep
      • +
      • maintainer turnover
      • +
      • collective decisions
      • +
    • +
    • UX is a fascinating problem (e.g. OS) +
        +
      • plug & play (intuition… about interfaces)
      • +
      • happy path (control)
      • +
      • defaults (expectations… about behavior)
      • +
    • +
    +
    +
    +
    +

    Stability vs Progress

    +
      +
    • 🤼 conflicting goals
    • +
    • ⚔️ breaking API can have large rippling effects +
        +
      • experience report of a facebook engineer on GHC upgrades
      • +
    • +
    • 🗼 small breakages add up +
        +
      • large projects have hundreds of dependencies
      • +
    • +
    • 🗣️ many discussions in the Haskell community +
        +
      • upgrade cost
      • +
      • language changes (Haskell report)
      • +
      • academic background of Haskell (academia vs industry?)
      • +
      • the role of committees
      • +
    • +
    • ⚖️ how to strike a balance? +
        +
      • SemVer does not solve it (why?)
      • +
    • +
    +
    +
    +

    Composition

    +
      +
    • 💜 I love small programs
    • +
    • categories of composition +
        +
      • λ: functions
      • +
      • 📚 libraries (the issue with types)
      • +
      • ⚙️ programs
      • +
    • +
    • unix philosophy +
        +
      • 🛠️ do one thing and do it well
      • +
      • ⚗️ pipes, compose stdout and stdin (re-usable)
      • +
    • +
    • how to make your project composable? +
    • +
    +
    +
    +

    Questions/Arguments?

    +

    +
    diff --git a/nus.md b/nus.md index 9f9cd68..da168c8 100644 --- a/nus.md +++ b/nus.md @@ -1,6 +1,24 @@ -% Two decades of Open Source -% Julian Ospald -% Sep 20, 2024 +--- +output: + slidy_presentation: + duration: 45 +title: Two decades of Open Source +author: Julian Ospald +date: Sep 20, 2024 +--- + +## Follow the presentation {.centered} + +![](QR.png){#id .class height=500px} + +## Structure of this talk + +1. Introduction (about me and my career) +2. Open Source (what it is and its value) +3. First chapter: Gentoo and package management +4. Second chapter: GHCup +5. Third chapter: Haskell Core Libraries +6. Lessons learned # Introduction @@ -66,6 +84,8 @@ * users * collaborators * 🕸️ network effects + * OSS community adopts fast + * compare with git ## Reality of Open Source @@ -80,7 +100,7 @@ * will stop maintenance at some point * don't care much about their users -# Gentoo and package management +# First chapter: Gentoo and package management ## What is Gentoo @@ -146,7 +166,7 @@ src_install() { - shipping a "chain" instead of a single artifact * high impact on small mistakes (e.g. assuming a specific shell) -## Packaging challenges (pt 2.) +## Packaging challenges (pt. 2) * communication between teams/maintainers * execution of large changes @@ -166,18 +186,19 @@ src_install() { * combining components to a coherent system (init system, coreutils, kernel, ...) * a choice of **defaults** -## Why Gentoo made me a better programmer +## Programming lessons * primary packaging skill: being meticulous * small mistakes -> big impact + * being as precise as possible about what you want to achieve * long term maintenance of small code pieces * intense review culture * strict policies and workflow guidelines * how to learn complex system -# GHCup +# Second chapter: GHCup -## Demo +## Demo {.centered} ![](ghcup.png){#id .class height=500px} @@ -232,14 +253,6 @@ curl -s -L \ - VSCode, stack, cabal-install integration * ![](ghaction.png){#id .class width=32 height=32px} CI provisioning (e.g. github actions) -## The difference to Gentoo - -* one-man project (mostly) -* much tighter coupling between upstream (e.g. GHC developers) and downstream (GHCup developers) - * heavier on relationship issues -* much more responsibility -* position of authority - ## Relationships in detail Dependencies: @@ -255,7 +268,7 @@ Dependencies: - platform support - binary distributions (the `.tar.gz`/`.zip`) -## Relationships in detail +## Relationships in detail (pt. 2) Dependents: @@ -269,3 +282,179 @@ Dependents: - 🧰 tools - [vscode-haskell](https://github.com/haskell/vscode-haskell), [Haskell playground](https://play.haskell.org/), [nvim-lsp-installer](https://github.com/williamboman/nvim-lsp-installer) +## Programming lessons + +* writing a small single-purpose program from scratch +* how to design command line interfaces +* high impact of decisions (not just mistakes) + - bugs now affect GitHub CI and companies + - can make "Haskell" look bad +* no one to review + - => review your own code +* constantly thinking about ways to improve reliability + - can't rely on anyone else to catch bugs + +## The difference to Gentoo + +Both deal with installation, but... + +* more code to maintain (not just packaging) for me +* one-man project (mostly) +* much tighter coupling between upstream (e.g. GHC developers) and downstream (GHCup developers) + * heavier on relationship issues +* less dependencies, but much more responsibility +* position of authority + - what to consider? +* most of my work today is support + +# Third Chapter: Haskell Core Libraries + +## What are Haskell Core Libraries? + +* bundled with the compiler +* fundamental building blocks (primitives) +* base library + - available to all programs by default + - contains the "Prelude" (standard library) + +## Core libraries I maintain + +* filepath +* unix +* os-string +* file-io + +## Challenges + +* changes are extremly expensive +* writing good primitives is hard (non-specific APIs) +* lots of odd knowledge + - e.g. Windows filepaths + - `C:foo` + - `/bar` + - `\\?\GLOBALROOT\Device\Harddisk0\Partition2\foo\bar` + - Posix standard +* portability + +## Core Libraries Committee + +* 7 members +* manages API changes of `base` only + * requires a proposal + * requires impact assessment for breaking changes + * requires an up-front implementation of the change +* ensures other core libraries have active maintainers + * does not interfere with maintenance + +## Driving changes across core libraries (case study) + +Abstract FilePath Proposal: + +- Haskell String type: `type String = [Char]` + - `Char` is a unicode code point + - not bytes + - is interpreted (decoded) + - depends on locale +- affects most core libraries +- implement as a breaking change (base), or... + - in "user-space" +- lack of higher authority + - building consensus + - convincing multiple maintainers + - patching many libraries +- open source politics + +## Programming lessons + +* how to design good primitives + - as opposed to abstractions +* considering every impact of API changes +* doing history research on past design choices + - important design decisions may not be documented + - may look innocent + - chaning them might be devastating + +# Lessons Learned + +## Collaboration + +::: incremental + +* main currency in Open Source is energy +* treat contributors like kings +* be mindful about boundaries (tricky balance) +* respect other projects workflows +* driving large changes requires + - consensus + - support + - a good value proposition + - a good execution plan (risk, breakage, ...) +* Haskell Foundation + +::: + +## Project maintenance + +::: incremental + +* dicatorships work, but are not sustainable +* plan for your departure +* bus factor is your constant enemy +* good decision making processes + - lightweight when risk is low + - elaborate when risk is high +* actively think about the contribution experience + - comment early +* how to maintain the project vision? + +::: + +## User Experience + +::: incremental + +* UX is harder than CS + * yet often an afterthought +* toolchains often lack a holistic UX vision +* UX vision gets easily lost in "maintenance mode" + * feature creep + * maintainer turnover + * collective decisions +* UX is a fascinating problem (e.g. OS) + - plug & play (intuition... about interfaces) + - happy path (control) + - defaults (expectations... about behavior) + +::: + +## Stability vs Progress + +* 🤼 conflicting goals +* ⚔️ breaking API can have large rippling effects + - experience report of a facebook engineer on GHC upgrades +* 🗼 small breakages add up + - large projects have hundreds of dependencies +* 🗣️ many discussions in the Haskell community + - upgrade cost + - language changes (Haskell report) + - academic background of Haskell (academia vs industry?) + - the role of committees +* ⚖️ how to strike a balance? + - SemVer does not solve it (why?) + +## Composition + +* 💜 I love small programs +* categories of composition + - λ: functions + - 📚 libraries (the issue with types) + - ⚙️ programs +* unix philosophy + - 🛠️ do one thing and do it well + - ⚗️ pipes, compose stdout and stdin (re-usable) +* how to make your project composable? + - [https://clig.dev/](https://clig.dev/) + +## Questions/Arguments? {.centered} + +![](jiddu.jpg){#id .class height=500px}