Topics:

A Discussion about Game Development

We were joined by Marcus Brown and Justin Walsh, the organizers of the Phoenix Unreal Engine Developers meetup. Both are excited about Rust and it's potential for game development.

Rust's safety is attractive. Marcus described how tooling in C++ provides safety but doesn't optimize code. Venturing into the optimization space is where it gets dangerous. Of course there are tools like static analysis, but for large enough codebases (i.e. games) it doesn't catch everything.

Rust's performance is another benefit. The Unreal Engine implements a garbage collection scheme that the developer has no direct control over. If there are a large number of items being prepared for deletion in a garbage collection cluster, it can cause a hitch (or hitched frame) which impacts performance. Perhaps some parts of the code could be implemented in Rust giving performance optimizations without the dangerous side effects.

Jacob notes that the best pattern for game development in Rust is ECS which runtime-checks all your stuff anyway, so you are not getting as much bang for the buck. An excellent talk by Catherine West covers this in detail.

rust-gpu and Embark Studios

Chris Miller brought up rust-gpu, a project to make Rust a first-class language and ecosystem for building GPU code. It is made by Embark Studios a games studio that uses Rust for development (perhaps the only one). They are aggresively hiring at the time this meetup occurred.

More Dark Forest Game via @jacobrosenthal and @BlaineBublitz

Dark Forest Toolbox Repo: https://github.com/phated/sophon

Sophon Tweeter Repo: https://github.com/jacobrosenthal/sophon_tweets

Dark Forest is an infinitely hackable real-time strategy idle-click game. It is based on zk-SNARKs (Zero-Knowledge Succinct Non-Interactive Argument of Knowledge).

Some examples of the infinite hackability of the game:

  • You can modify the client code. For instance, Blaine ripped out the tutorial code since they didn't need it and it was slow.

  • You don't technically need to pick a homeworld in the beginning of the game. Jacob and Blaine spent 4 hours just mining before they staked out homeworlds (though this didn't seem to give any advantage in the game)

  • You can explore outside the circle of the current known universe (since the radius of the universe expands as more players get added to it)

  • You can improve the efficiency of the miner's hash function to gain more information faster.

  • The Ethereum smart contract it is based on is not salted, so you could technically figure out stuff about the universe before the contract is even being used.

At the time of the meetup, Blaine and Jacob were hosting their client which has extended tools and attack notifications

They also made a project for automatically tweeting out stats from the public ethereum smart contract. The Twitter account is @sophon_eth. This repo uses tokio for async and was discussed in the next section of the meetup.

One final link from this talk was rust-web3, an Ethereum JSON-RPC multi-transport client built using Rust.

Learning Async Rust via @ChristopherSebastian

Chris needed to build a web frontend that could also fetch data from Redis and stream it to out to clients talking to an HTTP server. It would be simple to write in Go, but Chris wanted to use Rust because of the way it will integrate with other parts of the project which already are written in Rust. As he was building an HTTP server, Chris needed to write async code. He found that writing async in Rust was not as straightforward as it was writing it in C, so he wanted to learn more.

The Simplest Async Example Written in Rust

The quest started with finding the simplest example written in Rust that has zero dependencies. Jacob wrote a an example for the meetup. Chris found a more functional example which is only 44 lines of code. It's so small we can inline it here:

use std::sync::{Arc, Condvar, Mutex};
use std::task::{Context, Poll, RawWaker, RawWakerVTable, Waker};

#[derive(Default)]
struct Park(Mutex<bool>, Condvar);

fn unpark(park: &Park) {
    *park.0.lock().unwrap() = true;
    park.1.notify_one();
}

static VTABLE: RawWakerVTable = RawWakerVTable::new(
    |clone_me| unsafe {
        let arc = Arc::from_raw(clone_me as *const Park);
        std::mem::forget(arc.clone());
        RawWaker::new(Arc::into_raw(arc) as *const (), &VTABLE)
    },
    |wake_me| unsafe { unpark(&Arc::from_raw(wake_me as *const Park)) },
    |wake_by_ref_me| unsafe { unpark(&*(wake_by_ref_me as *const Park)) },
    |drop_me| unsafe { drop(Arc::from_raw(drop_me as *const Park)) },
);

/// Run a `Future`.
pub fn run<F: std::future::Future>(mut f: F) -> F::Output {
    let mut f = unsafe { std::pin::Pin::new_unchecked(&mut f) };
    let park = Arc::new(Park::default());
    let sender = Arc::into_raw(park.clone());
    let raw_waker = RawWaker::new(sender as *const _, &VTABLE);
    let waker = unsafe { Waker::from_raw(raw_waker) };
    let mut cx = Context::from_waker(&waker);

    loop {
        match f.as_mut().poll(&mut cx) {
            Poll::Pending => {
                let mut runnable = park.0.lock().unwrap();
                while !*runnable {
                    runnable = park.1.wait(runnable).unwrap();
                }
                *runnable = false;
            }
            Poll::Ready(val) => return val,
        }
    }
}

One thing that is different about Rust Futures compared to other languages: if the Future needs to be called again, it is the responsibility of the Future to reschedule itself inside the runtime. For futures to work at all there has to be a runtime that it registers itself with.

The Asynchronous Programming in Rust Book is a comprehensive guide to using Rust's async features.

Another great resource is Jon Gjengset's 4-hour video where he dives deep into Rust futures and async/await.

Picking an Async Runtime

In the Rust ecosystem, there are two major async runtimes, async-std and tokio. There is also smol, an async runtime without extra runtime-agnostic stuff (concurrency primitives, extension traits, etc.). Note that (async-std is now based on smol).

The Rustlang Community Wiki has a good overview of the Rust Async Ecosystem. To date, it seems like tokio is still the biggest player in the game and growing:

Async Market Growth Rates (1-month):
    tokio:    32000 --> 37000
    async-std: 6000 -->  7000
    smol:      2800 -->  2700

Smol

Smol is actually not small because it has a lot of dependencies, but the nice thing about it is that all those dependencies are written by the same guy, Stjepan Glavina. He was the person who made crossbeam channels, wrote the executor for async-std, and was also on the tokio team. So it is built on solid foundations. However, one worrying thing is that Stjepan deleted his blog including the post on why he made smol. There are unconfirmed reports that he is abandoning Rust for JavaScript 😜.

Smol's philosophy is: "Let's not keep reinventing API's, let's just write one and wrap synchronous things with it" (e.g. smol::Async<<std::net::TcpSocket>>).

Why use Smol? It can be split out into many crates. If you don't need one piece, you can just swap it out. For instance in embedded programming we can't use the smol executor because it uses std, so we swap that out for a different executor.

Tokio

Tokio is the oldest framework and grew up in the days before there was async support in std. As such, it doesn't try to match the std API. It has a lot of baggage but it is not going anywhere and is even growing in terms of market share.

Why use Tokio? If you are writing async and you don't know what you need, use Tokio: it's a good default and it has the whole kitchen sink.

Why NOT to use Tokio? If you are building something that someone else will be using, don't use tokio because you are forcing other people into the tokio ecosystem too. It affects everything it touches. Instead just use the futures crate and give people an async library without prescribing a runtime executor.

Async-std

Async-std tries to be 1:1 with the Rust std library. The philosophy is that if you learn sync Rust, your knowledge should transfer over (e.g. async_std::net::TcpSocket vs std::net::TcpSocket).

Tide is a minimal and pragmatic Rust web application framework that was written by the async-std team to showcase the use of async-std in an HTTP server. Many of the other web server frameworks are bought into the tokio framework.

Side Conversation on Good Learning Materials

Jon Gjengset is a great teacher and has awesome videos. For a projects-based approach to learning Rust, Chris liked the Learning Rust with Entirely Too Many Linked Lists Book as well as the Roguelike Tutorial in Rust Book (and another Roguelike Tutorial in Rust + tcod).

Crates You Should Know

  • rust-gpu: Project to make Rust a first-class language and ecosystem for building GPU code
  • futures: A library providing the foundations for asynchronous programming in Rust