An awesome library for retro video game cutscenes

Awesome Retro Video Game Cutscenes is a React library that took inspiration from old-school NES cutscenes (such as these). As of the time of this writing (05/2021), each scene accepts several commands:

• say a phrase
• draw-bg - draw a background
• jump to the next scene. Note that this is required; otherwise the story will end.
• delayInMs - wait X milliseconds before going to the next command
• filter - apply a PixiJS filter
• exportToVideo - under this hood, this uses the MediaRecorder API to save the cutscene to a webm video (chrome only).

You can use it to create art:

<AwesomeRetroCutscenes
  config={{
    background: {
      containerWidth: 200,
      containerHeight: 200,
    },
    dialog: {
      containerHeight: 0,
      containerWidth: 0,
    },
  }}
  scenes={[
    {
      title: "start",
      commands: [
        {
          "draw-bg": "/images/2021-05-03/destructured.png",
        },
        {
          filter: "glitch",
        },
        {
          delayInMs: 30000,
        },
      ],
    },
  ]}
/>

becomes...

You can use it to create comics (use the arrow keys to navigate between scenes):

<AwesomeRetroCutscenes
  config={{
    background: {
      containerWidth: 128,
      containerHeight: 128,
    },
    dialog: {
      containerHeight: 50,
      containerWidth: 128,
      textScrollSoundPath: "/sounds/textscroll.wav",
    },
  }}
  scenes={[
    {
      title: "start",
      commands: [
        {
          "draw-bg": "/images/2021-05-27/01.png",
        },
        {
          say: "GET OUT OF MY HOUSE!!!",
          actor: "mom",
        },
        {
          delayInMs: 2000,
        },
        {
          jump: "02",
        },
      ],
    },
    {
      title: "02",
      commands: [
        {
          "draw-bg": "/images/2021-05-27/02.png",
        },
        {
          delayInMs: 2000,
        },
        {
          jump: "03",
        },
      ],
    },
    {
      title: "03",
      commands: [
        {
          "draw-bg": "/images/2021-05-27/03.png",
        },
        {
          say: "Not you sweetie...",
          actor: "mom",
        },
        {
          delayInMs: 2000,
        },
        {
          jump: "04",
        },
      ],
    },
    {
      title: "04",
      commands: [
        {
          "draw-bg": "/images/2021-05-27/04.png",
        },
        {
          say: "...the puppy!",
          actor: "mom",
        },
      ],
    },
  ]}
/>

becomes...

Note: you must give the comic focus for sound (browser limitation).

Next Steps

• scene transitions!
• music!

The Inspiration

The year was 2020 and I was under quarantine :-). I had been dabbling in HTML5 game development using GDevelop. I was finding it hard to program game logic using its "visual programming UI"---my software-engineer background was getting in the way.

I stumbled on PixiJS, a low-level JS library to create "digital content" (including games). "It seemed perfect," I said to myself. "Seems to hit the right sweet spot between performance and adaptability. It has traction, a large community, good docs..."

But then I took a look at the examples. Boy where they something out of 1990. Imagine taking the entire app state and shoving it in the this context, and then mutating the same subslice across several functions. Yea, a shit show.

And there's the question of React integration. I found React Pixi but my gut told me that making PixiJS declarative (the right way) would require a complete rewrite. In other words, I had questions about performance and adaptability.

So I decided to write my own React glue.

Around the same time, I was pivoting from a game to some kind of "interactive picture book/comic". And so Awesome Retro Video Game Cutscenes was born!

Some More Thoughts on a Declarative PixiJS

If you need to integrate the digital content with a React app, then most likely you'll need a bridge between app state and PixiJS state.

But instead of creating a React component wrapper for every PixiJS class, it's more scalable to create custom adhoc wrappers as needed.

For example, this component exposes the bunny texture's x position so that it can be managed by the app. (Note that this code is untested---it's more of an idea than working code.)

function BasicDemo(props: { x: number; container: PIXI.Container }) {
  const texture = useRef(PIXI.Texture.from("examples/assets/bunny.png"));
  const bunny = useRef(new PIXI.Sprite(texture));

  useEffect(
    function addToComponent() {
      container.addChild(bunny.current);

      return () => {
        container.removeChild(bunny.current);
      };
    },
    [container]
  );

  const ticker = usePixiTicker();

  useEffect(
    function animateX() {
      ticker.addOnce(() => {
        bunny.current.x = x;
      });
    },
    [ticker, x]
  );
}

Note how the component accepts its container as a property. This is problematic (since it implies prop drilling). React Context won't work either since the container can be dynamic.

Off the top of my head, one solution is to put all Containers in a giant Context and then use the upcoming useSelectedContext to avoid rendering issues.

Awesome Retro Video Game Cutscenes doesn't follow the above approach and assumes a relatively flat hierarchy of objects.