Gatsby JS stands on the shoulders of thousands

Many years ago Linux founder Linus Torvalds borrowed Sir Isaac Newton’s “standing on the shoulders of giants” phrase when discussing his open source work with Linux. While it’s a nice sentiment — “Hey, I’ve only been able to do great work because of a few others’ great work” — that may have applied to Linux in 1991, it doesn’t do a good job of describing open source in 2020. When someone releases open source code today, after all, they’re drawing on much more than a tiny set of genius developers.

No, as the latest release of Gatsby.js shows, modern open source projects depend on thousands of other projects. Or, as Gatsby founder Kyle Mathews put it in an interview, “We’re standing on the shoulders of thousands of [ordinary] people.”

1000x faster builds? Yes, please!

In the beginning (of the web) was the static site generator. Over time, as Mathews described in his post, organizations turned to database-driven websites to support increasingly dynamic features. As websites grew, static site generation resulted in slow build speeds, even as database-driven tools like WordPress grew in favor with the marketing folks that didn’t want to bother coding in Markdown. Pre-rendering helped the site generators, but not enough to make them the preferred option for large sites.

But that was then; this is now.

Earlier in 2020 Gatsby introduced Gatsby Builds, which made build times 60X faster than standard continuous deployment solutions through a clever combination of distributed computing and sophisticated caching features. Nice. But with the recent release of Incremental Builds, Gatsby is getting builds under 10 seconds for data edits, which can represent a 1000x improvement over existing build solutions.

How? “The biggest thing we’ve done,” said Mathews in an interview, “is created a dependency tracking system that allows us to cheaply figure out what needs updating between builds.” If this sounds like the approach taken by build tools like Bazel, or data processing tools like Apache Spark and Apache Flink, it’s because it is. “Gatsby treats website change events from data/code like other stream processors—we figure out what’s changed and cheaply update it.” Traditional static site generators have long had a batch processing approach to data, but Gatsby’s new approach is real-time stream processing.

Source link