Open-source News

Why we chose the Clojure programming language for Penpot Andrey Antukh Sun, 08/07/2022 - 03:00 Register or Login to like Register or Login to like

"Why Clojure?" is probably the question we've been asked the most at Penpot. We have a vague explanation on our FAQ page, so with this article, I'll explain the motivations and spirit behind our decision.

It all started in a PIWEEK. Of course!

During one of our Personal Innovation Weeks (PIWEEK) in 2015, a small team had the idea to create an open source prototyping tool. They started work immediately, and were able to release a working prototype after a week of hard work (and lots of fun). This was the first static prototype, without a backend.

I was not part of the initial team, but there were many reasons to choose ClojureScript back then. Building a prototype in just a one-week hackathon isn't easy and ClojureScript certainly helped with that, but I think the most important reason it was chosen was because it's fun. It offered a functional paradigm (in the team, there was a lot of interest in functional languages).

It also provided a fully interactive development environment. I’m not referring to post-compilation browser auto-refresh. I mean refreshing the code at runtime without doing a page refresh and without losing state! Technically, you could be developing a game, and change the behavior of the game while you are still playing it, just by touching a few lines in your editor. With ClojureScript (and Clojure) you don't need anything fancy. The language constructs are already designed with hot reload in mind from the ground up.

Image by:

(Andrey Antukh, CC BY-SA 4.0)

I know, today (in 2022), you can also have something similar with React on plain JavaScript, and probably with other frameworks that I’m not familiar with. Then again, it's also probable that this capability support is limited and fragile, because of intrinsic limitations in the language, sealed modules, the lack of a proper REPL.

About REPL

In other dynamic languages, like JavaScript, Python, Groovy (and so on), REPL features are added as an afterthought. As a result, they often have issues with hot reloading. The language patterns are fine in real code, but they aren't suitable in the REPL (for example, a const in JavaScript evaluates the same code twice in a REPL).

These REPLs are usually used to test code snippets made for the REPL. In contrast, REPL usage in Clojure rarely involves typing or copying into the REPL directly, and it is much more common to evaluate small code snippets from the actual source files. These are frequently left in the code base as comment blocks, so you can use the snippets in the REPL again when you change the code in the future.

In the Clojure REPL, you can develop an entire application without any limitations. The Clojure REPL doesn't behave differently from the compiler itself. You're able to do all kinds of runtime introspection and hot replacement of specific functions in any namespace in an already running application. In fact, it's not uncommon to find backend applications in a production environment exposing REPL on a local socket to be able to inspect the runtime and, when necessary, patch specific functions without even having to restart the service.

Programming and development Red Hat Developers Blog Programming cheat sheets Try for free: Red Hat Learning Subscription eBook: An introduction to programming with Bash Bash shell scripting cheat sheet eBook: Modernizing Enterprise Java From prototype to the usable application

After PIWEEK in 2015, Juan de la Cruz (a designer at Penpot, and the original author of the project idea) and I started working on the project in our spare time. We rewrote the entire project using all the lessons learned from the first prototype. At the beginning of 2017, we internally released what could be called the second functional prototype, this time with a backend. And the thing is, we were still using Clojure and ClojureScript!

The initial reasons were still valid and relevant, but the motivation for such an important time investment reveals other reasons. It's a very long list, but I think the most important features of all were: stability, backwards compatibility, and syntactic abstraction (in the form of macros).

Image by:

(Andrey Antukh, CC BY-SA 4.0)

Stability and backwards compatibility

Stability and backwards compatibility are one of the most important goals of the Clojure language. There's usually not much of a rush to include all the trendy stuff into the language without having tested its real usefulness. It's not uncommon to see people running production on top of an alpha version of the Clojure compiler, because it's rare to have instability issues even on alpha releases.

In Clojure or ClojureScript, if a library doesn't have commits for some time, it's most likely fine as is. It needs no further development. It works perfectly, and there's no use in changing something that functions as intended. Contrarily, in the JavaScript world, when you see a library that hasn't had commits in several months, you tend to get the feeling that the library is abandoned or unmaintained.

There are numerous times when I've downloaded a JavaScript project that has not been touched in 6 months only to find that more than half of the code is already deprecated and unmaintained. On other occasions, it doesn’t even compile because some dependencies have not respected semantic versioning.

This is why each dependency of Penpot is carefully chosen, with continuity, stability, and backwards compatibility in mind. Many of them have been developed in-house. We delegate to third party libraries only when they have proven to have the same properties, or when the effort to time ratio of doing it in-house wasn't worth it.

I think a good summary is that we try to have the minimum necessary external dependencies. React is probably a good example of a big external dependency. Over time, it has shown that they have a real concern with backwards compatibility. Each major release incorporates changes gradually and with a clear path for migration, allowing for old and new code to coexist.

Syntactic abstractions

Another reason I like Clojure is its clear syntactic abstractions (macros). It's one of those characteristics that, as a general rule, can be a double-edged sword. You must use it carefully and not abuse it. But with the complexity of Penpot as a project, having the ability to extract certain common or verbose constructs has helped us simplify the code. These statements cannot be generalized, and the possible value they provide must be seen on a case-by-case basis. Here are a couple of important instances that have made a significant difference for Penpot:

• When we began building Penpot, React only had components as a class. But those components were modeled as functions and decorators in a rumext library. When React released versions with hooks that greatly enhanced the functional components, we only had to change the implementation of the macro and 90% of Penpot's components could be kept unmodified. Subsequently, we've gradually moved from decorators to hooks completely without the need for a laborious migration. This reinforces the same idea of the previous paragraphs: stability and backwards compatibility.
• The second most important case is the ease of using native language constructs (vectors and maps) to define the structure of the virtual DOM, instead of using a JSX-like custom DSL. Using those native language constructs would make a macro end up generating the corresponding calls to React.createElement at compile time, still leaving room for additional optimizations. Obviously, the fact that the language is expression-oriented makes it all more idiomatic.

Here's a simple example in JavaScript, based on examples from React documentation:

function MyComponent({isAuth, options}) {
    let button;
    if (isAuth) {
        button = <LogoutButton />;
    } else {
        button = <LoginButton />;
    }

    return (
        <div>
          {button}
          <ul>
            {Array(props.total).fill(1).map((el, i) =>
              <li key={i}>{{item + i}}li>
            )}
          ul>
        div>
    );
}

Here's the equivalent in ClojureScript:

(defn my-component [{:keys [auth? options]}]
  [:div
   (if auth?
     [:& logout-button {}]
     [:& login-button {}])
   [:ul
    (for [[i item] (map-indexed vector options)]
      [:li {:key i} item])]])

All these data structures used to represent the virtual DOM are converted into the appropriate React.createElement calls at compile time.

The fact that Clojure is so data-oriented made using the same native data structures of the language to represent the virtual DOM a natural and logical process. Clojure is a dialect of LISP, where the syntax and AST of the language use the same data structures and can be processed with the same mechanisms.

For me, working with React through ClojureScript feels more natural than working with it in JavaScript. All the extra tools added to React to use it comfortably, such as JSX, immutable data structures, or tools to work with data transformations, and state handling, are just a part of the ClojureScript language.

Guest Language

Finally, one of the fundamentals of Clojure and ClojureScript ​​is that they were built as Guest Languages. That is, they work on top of an existing platform or runtime. In this case, Clojure is built on top of the JVM and ClojureScript on top of JavaScript, which means that interoperability between the language and the runtime is very efficient. This allowed us to take advantage of the entire ecosystem of both Clojure plus everything that's done in Java (the same is true for ClojureScript and JavaScript).

There are also pieces of code that are easier to write when they're written in imperative languages, like Java or JavaScript. Clojure can coexist with them in the same code base without any problems.

There's also an ease of sharing code between frontend and backend, even though each one can be running in a completely different runtime (JavaScript and JVM). For Penpot, almost all the most important logic for managing a file's data is written in code and executed both in the frontend and in the backend.

Perhaps you could say we have chosen what some people call a "boring" technology, but without it actually being boring at all.

Trade-offs

Obviously, every decision has trade-offs. The choice to use Clojure and ClojureScript is not an exception. From a business perspective, the choice of Clojure could be seen as risky because it's not a mainstream language, it has a relatively small community compared to Java or JavaScript, and finding developers is inherently more complicated.

But in my opinion, the learning curve is much lower than it might seem at first glance. Once you get rid of the it's different fear (or as I jokingly call it: fear of the parentheses), you start to gain fluency with the language very quickly. There are tons of learning resources, including books and training courses.

The real obstacle I have noticed is the paradigm shift, rather than the language itself. With Penpot, the necessary and inherent complexity of the project makes the programming language the least of our problems when facing development: building a design platform is no small feat.

This article originally appeared on the Kaleidos blog and has been republished with permission. 

Though it is not a mainstream language, Clojure was the right language for Penpot to choose because of its key features: stability, backwards compatibility, and syntactic abstraction.

Art and design Programming What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

New IoT RapperBot Malware Targeting Linux Servers via SSH Brute-Forcing Attack  The Hacker News

Takashi Iwai of SUSE as the Linux sound subsystem maintainer has submitted all the new hardware support and feature updates targeting the Linux 6.0 merge window. The Linux 6.0 sound driver changes are notable when it comes to new AMD and Intel hardware support among other changes...

NetBSD 9.3 has been released as the newest version of this open-source BSD operating system known for running on many diverse platforms thanks to its focus oncode portability...

New GwisinLocker ransomware encrypts Windows and Linux ESXi servers  BleepingComputer

For those with a newer ASUS gaming laptop boasting RGB lighting for the keyboard, that functionality could soon be working nicely thanks to work happening within the Linux kernel and the open-source Asusctl project...

ASUS Linux Driver Gets Patches For RGB Keyboard Controls  Phoronix

Going back to late 2020 there has been bits of Intel Vulkan ray-tracing preparations landing within their Mesa "ANV" open-source Vulkan driver in anticipation of Xe HPG with hardware ray-tracing capabilities...

Intel's Open-Source Vulkan Driver Moves Closer To Working Ray-Tracing Support  Phoronix

Support for old NEC VR4100 CPUs based on the MIPS R4000 core is being removed with the Linux 6.0 kernel leading to devices like the old IBM WorkPad Z50 no longer being supported...

Linux 6.0 Drops Support For Old NEC VR4100 MIPS CPUs Found In The IBM WorkPad & More  Phoronix

KDE developers have started the month of August to a lot of fixes and polishing for the Plasma desktop...

OpenJ9 v0.33 was released on Friday as the newest version of this Eclipse Foundation Java Virtual Machine (JVM) that was formerly developed by IBM...

Old-school technical writing with groff Jim Hall Sat, 08/06/2022 - 03:00 Register or Login to like Register or Login to like

One of my favorite stories about Unix is how it turned into a text processing system. Brian Kernighan tells the story in his book Unix: A History and a Memoir (chapter 3) but to summarize: The Unix team at Bell Labs ran the original Unix on a PDP-7 computer, but it was a tiny system and didn't have sufficient resources to support new work. So Ken Thompson and others lobbied to purchase a new PDP-11 computer. Management denied the request. Around the same time, the Patents department planned to buy a new computer platform to produce patent applications using proprietary document formatting software. The Unix group proposed that the Patents department instead buy a new PDP-11 for the Unix team, and the Unix team would create formatting software for them.

That new formatting system was called nroff, short for "New Roff," an updated version of a text formatting program called Roff from a 1960s computer system. The name Roff came from the old expression, "I'll run off a document."

Basic formatting with nroff

By default, nroff collects words and fills paragraphs. When nroff encounters a blank line, it starts a new paragraph. For example, start with this article's introduction, which is only a few paragraphs long:

$ cat intro
Old-school technical writing with groff
Jim Hall
 
One of my favorite stories about Unix is how it turned
into a text processing system. Brian Kernighan tells the
story in his book Unix: A History and a Memoir (chapter 3)
but to summarize:
The Unix team at Bell Labs ran the original Unix on
a PDP-7 computer, but it was a tiny system and didn't
have sufficient resources to support new work. So Ken
Thompson and others lobbied to purchase a new PDP-11
computer. Management denied the request. Around the same
time, the Patents department planned to buy a new computer
platform to produce patent applications using proprietary
document formatting software. The Unix group proposed
that the Patents department instead buy a new PDP-11 for
the Unix team, and the Unix team would create formatting
software for them.
 
That new formatting system was called nroff, short for
"New Roff," an updated version of a text formatting program
called Roff from a 1960s computer system. The name Roff
came from the old expression, "I'll run off a document."

If you process this file with nroff, lines are "glued" together so the output is paragraphs with full justification. Using nroff also hyphenates words, if that helps balance lines in the text:

$ nroff intro | head
Old‐school technical writing with groff Jim Hall
 
One  of  my  favorite  stories about Unix is how it turned into a
text processing system. Brian Kernighan tells the  story  in  his
book  Unix:  A History and a Memoir (chapter 3) but to summarize:
The Unix team at Bell Labs ran the original Unix on a PDP‐7  com‐
puter,  but  it  was a tiny system and didn’t have sufficient re‐
sources to support new work. So Ken Thompson and  others  lobbied
to purchase a new PDP‐11 computer. Management denied the request.
Around the same time, the Patents department planned to buy a new

Original Unix systems used a typewriter-style printer that used 66 lines of 80 columns on a US Letter page, and nroff makes the same assumptions. It also adds empty lines so each page of output is 66 lines per page, but I've used the head command to show just the first few lines of output because my sample text isn't very long.

Breaking lines and centering text

The first two lines were meant to be separate lines of text. You can insert a formatting instruction to tell nroff to add a line break. All nroff instructions start with a dot, followed by a brief command. To add a line break, use the .br instruction between the first and second line:

Old-school technical writing with groff .br Jim Hall

When you process this new file, nroff prints the title and author on separate lines:

$ nroff intro | head
Old‐school technical writing with groff
Jim Hall
 
One  of  my  favorite  stories about Unix is how it turned into a
text processing system. Brian Kernighan tells the  story  in  his
book  Unix:  A History and a Memoir (chapter 3) but to summarize:
The Unix team at Bell Labs ran the original Unix on a PDP‐7  com‐
puter,  but  it  was a tiny system and didn’t have sufficient re‐
sources to support new work. So Ken Thompson and  others  lobbied
to purchase a new PDP‐11 computer. Management denied the request.

You can add other formatting to make this document look better. To center the top two lines, use the .ce formatting request. This takes a number argument, to indicate how many lines nroff should center. Here, you can center the top two output lines with the .ce 2 request:

.ce 2 Old-school technical writing with groff .br Jim Hall

With this added instruction, nroff correctly centers the first two lines:

$ nroff intro | head
             Old‐school technical writing with groff
                            Jim Hall
 
One  of  my  favorite  stories about Unix is how it turned into a
text processing system. Brian Kernighan tells the  story  in  his
book  Unix:  A History and a Memoir (chapter 3) but to summarize:
The Unix team at Bell Labs ran the original Unix on a PDP‐7  com‐
puter,  but  it  was a tiny system and didn’t have sufficient re‐
sources to support new work. So Ken Thompson and  others  lobbied
to purchase a new PDP‐11 computer. Management denied the request.

More Linux resources Linux commands cheat sheet Advanced Linux commands cheat sheet Free online course: RHEL technical overview Linux networking cheat sheet SELinux cheat sheet Linux common commands cheat sheet What are Linux containers? Our latest Linux articles Adding page margins

Printing this to a printer results in text starting on the first line of the page, and against the left edge. To add a few lines of extra space from the top of the page, use the .sp request, with the number of blank lines to add:

.sp 5
.ce 2
Old-school technical writing with groff
.br
Jim Hall

By default, nroff formats the output so each line is 65 columns wide. Printing to an 80 column US Letter page leaves 15 empty columns. Adding 7 spaces on the left side neatly balances the output with equal left and right page margins. You can create this page offset using the .po 7 request:

.po 7
.sp 5
.ce 2
Old-school technical writing with groff
.br
Jim Hall

Processing the new file with nroff produces a plain text page that's ready to print:

$ nroff intro | head Old‐school technical writing with groff Jim Hall One of my favorite stories about Unix is how it turned into a text processing system. Brian Kernighan tells the story in hisPrinting to a laser printer

Later, the Unix team at Bell Labs acquired a phototypesetting machine, capable of producing printed text similar to a laser printer. To support the typesetter's new capabilities, the Unix team updated nroff to become the typesetter-specific troff program, and a few years later updated it again to become ditroff, the device-independent version of troff.

Linux systems provide modern versions of nroff and troff using the GNU groff program. You can still use the old nroff program name to generate plain text output, or troff to produce ditroffcompatible output. Using the groff program, you can also prepare documents for other kinds of output files, such as Postscript.

You can process the same input file using groffto print on a Postscript-compatible laser printer by selecting a suitable output type using the -T option, such as -Tps to generate a Postscript file. For example, I can print to a printer with the lpr command and the HP_LaserJet_CP1525nw device, because that's how my Linux system recognizes my laser printer:

$ groff -Tps intro | lpr -P "HP_LaserJet_CP1525nw"Generating other kinds of output

If you instead want to save the output as a PDF file, you can convert the Postscript using the ps2pdf tool:

$ groff -Tps intro | ps2pdf - > intro.pdf

To generate a web page from the same file, use -Thtml to set the output type to HTML:

$ groff -Thtml intro > index.html

The groff command supports lots of other built-in formatting requests to provide other kinds of document formatting. If you want to learn the other default formatting requests available to you in the GNU groff implementations of nroff and troff, refer to chapter 5 in the The GNU Troff Manual.

Formatting documents using these built-in commands takes a lot of effort to keep everything looking the same. Technical writers who use groff instead use a collection of formatting requests called macros, which provide their own commands to generate section headings, paragraphs, block quotes, footnotes, lists, and other useful document formatting. To learn more about one popular macro package, read How to format academic papers on Linux with groff -me on Opensource.com.

Take a trip back in time to experience text formatting from a bygone era.

Image by:

LSE Library. Modified by Opensource.com. CC BY-SA 4.0

Linux What to read next This work is licensed under a Creative Commons Attribution-Share Alike 4.0 International License. Register or Login to post a comment.

OpenRISC Enables PCI Support With Linux 6.0  Phoronix

While OpenRISC has been around a decade longer than RISC-V and its original support in the Linux kernel dates back to the v3.1 days, on the hardware side OpenRISC hasn't enjoyed nearly as much success as RISC-V and its kernel support not advancing nearly as rapidly. Now with Linux 6.0, OpenRISC is finally exposing PCI bus support...

How to Password-Protect Files and Folders on Linux  MUO - MakeUseOf

How to Password-Protect Files and Folders on Linux  MUO - MakeUseOf

Proton 7.0-4 is coming to bring more compatibility to Linux and Steam Deck  GamingOnLinux

Linux Botnet Targets Weak SSH Server Credentials  Duo Security