What's new with Scala Native?

Friday 26 May 2017

Martin Duhem, Guillaume Massé and Denys Shabalin

It has been a little bit more than two months since Scala Native 0.1 has been released. What’s new in Scala Native? What should you expect for future releases?

Scala Native 0.2

In April, Scala Native 0.2 was released. The main focus of this release was to increase the coverage of classes from the JDK, such that more programs can be ported to Scala Native without any further effort. What parts of Java do we now support in Scala Native? Lots! We’ve added support for IO and regular expressions, among others:

Improvements to the standard library

Support for file I/O APIs from java.io was added by @Duhemm from the Scala Center with help from @cedricviaccoz and @Korf74 in #574. Scala Native now supports enough to read and write files. Doing I/O with Scala Native feels just the same as in normal Scala or Java:
```
import java.io.{DataInputStream, File, FileInputStream}
val fis = new FileInputStream(new File("hello.txt"))
val dis = new DataInputStream(fis)
println(dis.readLine())
```
In #588, @MasseGuillaume from the Scala Center added support for regular expressions. This implementation relies on Google’s RE2 engine and uses a syntax slightly different from the JDK. Using regular expressions with Scala Native works similarly as it does on the JVM:
```
import java.util.regex._
val m = Pattern.compile("a+(b+)(a+)").matcher("aaabbba")
assert(m.find())
println(m.group(1)) // prints "bbb"
println(m.group(2)) // prints "a"
```

@densh added initial support for Scala’s Futures in #618, using an implementation similar to that of Scala.js, where Futures will be completed after the main method is executed. Here’s an example using Scala’s Futures with Scala Native. Of course, the same code works as well on the JVM:

import scala.concurrent.Future
import scala.concurrent.ExecutionContext.Implicits.global

object Test {
  def main(args: Array[String]): Unit = {
    println("Start")
    Future {
      println("Hello from the Future")
    }.foreach(_ => ())
    println("End")
  }
}

Scala Native now supports pointer subtraction. This work has been contributed by @jonas in #624. Pointer subtraction is useful, for instance, to determine how many elements there are between two elements of the same array:
```
val carr: Ptr[CChar] = toCString("abcdefg")
val cptr: Ptr[CChar] = string.strchr(carr, 'd')
println(cptr - carr) // prints '3'
```
@ekrich extended Scala Native’s implementation of String to support toUpperCase and toLowerCase in #573.
The implementation of java.lang.Character was extended to support Character.Subset and Character.UnicodeBlock by @densh in #651.

@asoltysik implemented support for system properties in #591.

println(System.getProperty("java.version"))   // prints '1.8'
println(System.getProperty("file.separator")) // '\' on Windows,
                                              // '/' elsewhere

Scala Native can now read environment variables using System.getEnv, thanks to @jonas’ efforts in #606.
stdin and stdout can reliably be read from and written to, thanks to @fduraffourg and @densh in #622 and #659.

Bugfixes

A bugfix has been contributed by @jonas to String.replace, fixing one broken benchmark at the same time. His work can be found in #616.
System.nanoTime was fixed by Brad Rathke in #549.
@xuwei-k fixed nativeAvailableDependencies which didn’t work in the test configuration in #565.

Improvements to the tooling and integration

@MasseGuillaume and @densh worked on refactoring Scala Native’s sbt plugin to make it more idiomatic in #568 and #630.
Follow up fixes were contributed by @jonas in #639 and #653. They improve how sbt determines the target architecture to compile to.

Preparing for a better garbage collector

In this release, @LukasKellenberger introduced in #539 a new setting in the sbt plugin that lets users select what implementation of the garbage collector should be used. Currently, it lets you select Boehm GC, or disable the garbage collector altogether.

This work was done in preparation for the improved GC that will be shipped with Scala Native 0.3!

A community effort

As shown, many of the improvements that were brought by Scala Native 0.2 have been contributed by members of the vibrant community that is developing itself around Scala Native. In total, to get to Scala Native 0.2, there have been 61 commits merged, 11,344 lines added and 1,954 lines deleted by 17 people: Denys Shabalin, Jonas Fonseca, Guillaume Massé, Martin Duhem, Lukas Kellenberger, Andrzej Sołtysik, Eric K Richardson, Remi Coudert, Florian Duraffour, Brad Rathke, Richard Whaling, Ruben Berenguel M, Sam Halliday, Shunsuke Otani, Cedric Viaccoz, Kenji Yoshida, Ignat Loskutov.

The combination of these improvements was enough to get a prototype of scalafmt running on Scala Native by @olafurpg, showing a blazing fast startup time!

@adriaanm @den_sh @scala_native OMG IT WORKS :D 30x faster! pic.twitter.com/M7V9udU5bT
— Ólafur Páll Geirsson (@olafurpg) April 27, 2017

What to expect for Scala Native 0.3?

The plans for the next release of Scala Native include a new garbage collector, a better integration with sbt and more additions to the standard library.

Improved garbage collector

The first releases of Scala Native use Boehm GC. A new and improved garbage collector has been under development by @LukasKellenberger and will be presented at Scala Days during Denys’ talk. Stay tuned for more more details to come soon!

The pull request introducing the new garbage collector can be found in #726.

Running tests from sbt

Currently, testing frameworks such as utest or scalacheck cannot be used with Scala Native. An effort to enable support for sbt-compatible testing frameworks has been undertaken by @Duhemm from the Scala Center and is expected to land in Scala Native’s third release.

Support for `java.nio`, `java.util.{jar, zip}`

The existing I/O capabilities of Scala Native have been extended in this release by adding support for the classes defined in the packages java.nio, java.util.jar and java.util.zip.

Smaller binaries

In #686, @densh started work to reduce the size of the binaries compiled by Scala Native, using a technique called selector-based row displacement.

These improvements make the dispatch table up to 10 times smaller, on some codebases in the wild.

0.4 and beyond

Some features are already in the works for Scala Native 0.4.

Windows support

@muxanick has been working on a port of Scala Native to Windows. The advancement of his work can be consulted in #691.

Automatic binding generation

A prototype of automatic binding generation is in development by @jonas. The goal is to be able to generate automatically bindings for C libraries. For instance, given the following C header test.h:

enum color {
  RED = 1,
  GREEN,
  BLUE = 100
};
typedef int SomeInt;
char *strchr(const char *s, int c);

We want to generate the following definitions:

@extern
object Test {
  object color {
    val RED = 1
    val GREEN = 2
    val BLUE = 100
  }
  type SomeInt = CInt
  def strchr(s: CString, c: CInt): CString = extern
}

Ultimately, this tool will make it much easier to providing bindings for C’s stdlib and external libraries.