ISO Emeritus, NTPSec
Senior Systems Analyst, IU CACR
@HedgeMage // http://security.engineering
Not yet C99 compliant.
Fragile build system.
Documentation between six and thirty years out of date.
Code locked up in a proprietary SCM system.
Technical debt dating back decades.
Vulnerability patches going public on a months-to-years response cycle.
Patches circulated in private weaponized and used to exploit servers across the internet.
Lack of access to development history made it difficult to audit the code and/or take on improvements as a drive-by contributor.
The overall state of the software, build infrastructure, and community made NTP brittle, full of vulnerabilities, and difficult to improve.
No OSS gets broken to the point of crisis without a driving set of systemic social problems. If these are not addressed, any repair will be short-term, as the underlying cause of the original technical problems will continue to cause new technical problems.
Bringing Order to Chaos
Fixing bugs is temporary. More bugs are coming.
Long-term impact comes from making bugs easier to fix, and eliminating or preventing classes of bugs.
A good rescue results in a long tail of bug fixing.
Familiar with ancient C code
Experienced in Linux/UNIX systems programming
Capable of working on highly critical code
With some idea how time works
Who care about open source and security
Who can spend a lot of time on this.
A way to keep those programmers fed
Help with documentation and toolchain work
Means to demonstrate to the existing NTP community that we weren't abandoning them
An understanding of the existing install base that we didn't have
The means to maintain the code, documentation, and community post-rescue
Some way to convince people to actually deploy the thing
Harlan Stenn - NTP Classic Maintainer
Adam Nuwer - Volunteer Sysadmin, Community Member
Von Welch - My Boss, CACR Director, CTSC PI
Anita Nikolich - NSF PM for CTSC
Members of the NTP Classic Community
Tim Minick - then of Gemini Observatory
Eric Raymond - (yes, that ESR) GPSd maintainer, Software Architect
Gary Miller -- GPSd Software Architect
Amar Takhar - former NTP Classic team member, build system geek
Leslee Cooper - CACR Admin Director, got me an awesome student intern (NaLette Brodnax) for docs work!
Many, many people who answered nosy questions about their NTP usage.
Mark Atwood -- Took the handoff as NTPSec Project Manager
Daniel Franke -- Took the handoff in as NTPSec ISO
Many other people I've failed to name.
Two administrative staff.
2-4 semi-active community members.
Susan Sons, PM / ISO
Eric Raymond, lead dev
Gary Miller, developer
NaLette Brodnax, docs
Amar Takhar. tools dev
...and a handful of concerned community members.
NTPSec's core team has been through a lot, but we still meet up about once a year and hang out, because it was a wild ride with good people. I was given an emeritus title when I stepped down last spring, in the hope that I'd remain "part of the family".
How many currently active committers account for >50% of the code base?
Breakdown by Dave Nalley:
Image credit: Dave Nalley
This deck is at: https://slides.com/hedgemage/savingtime
To Wikimedia Foundation for their awesome library of freely reusable media, which spared you from my toddler-like drawing ability.
To Indiana University's Center for Applied Cybersecurity Research, and specifically the NSF-funded Center for Trustworthy Scientific Cyberinfrastructure, who funded the NTP Rescue project. Also to the Internet Civil Engineering Institute, who aided with organization and developer resources.
To O'Reilly, for bringing me here to tell you this story.
To the NTP Security Project team, who made sure the rescue effort didn't go to waste. NTPSec is poised to replace NTP classic in the coming year in installations around the world.
To the countless individual humans along the way who did NOT say
"this is somebody else's problem".
Saving Time by Susan Sons is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Permissions beyond the scope of this license may be available; send inquiries to firstname.lastname@example.org.