In his wonderful essay on Mother 3, one of the short chapters is called “Give my regrets to the next frog you meet”. He writes:
The frogs are the way one saves progress in the game, but while this is their function within the logic of the video game the experience of initiating dialogue with a frog is, more literally, that the character one controls is locked in place and a black box pops up with rapidly scrolling dialogue in white. According to convention the player understands that this text is supposed to be being spoken by the frog, but the frogs themselves are tiny little collections of at most thirty greenish-pixels and one cannot see their mouths moving. The sound that comes out of the tinny GameBoy Advance speakers is a bleeping which has not been programmed to sound like a ribbit, but since we expect that it should have been we often hear it as if it has. Many of the frogs will say “give my regards to the next frog you meet”, but not all of them. Some of them, in literal fact, say “give my regrets to the next frog you meet”, and the eponymous correction I must give here is that Max is wrong to believe he was consistently misreading this.
I was puzzled about how this had happened—what is it about the experience of Mother 3 that has my friend questioning his eyes in this way?—until another friend of mine told me that he might have been playing the game through a Nintendo DS. The confusion appears to lie in the fascinating technology of gaze tracking. Mother 3 was originally written for a release on the Super Famicom, and then the Nintendo 64, before it was delayed for a decade and pushed onto the GameBoy Advance handheld console. When it was eventually released in 2006 on that platform the newer Nintendo DS had already been out for two years, and there was a good deal of debate within the development team and the publishing directors at Nintendo about whether the game should be ported once again, but ultimately this was shot down because no one could be bothered to think of a use for the two screens which the Nintendo DS used as its unique selling point. Only, in the midst of this debate, development on the game continued; one programmer made use of the front facing camera of the DS to implement a gaze tracker. Looking for a single reflective pixel of white in each of the player’s pupils, the game could pinpoint the exact position on the screen they were looking at. In the GameBoy version the frogs flip, basically at random, from one frog to the next, between wishing their regards and their regrets on the next frog the player meets. But the sole surviving use of that strange piece of code in the game is that, when one plays on the Nintendo DS, the black box with the white text in it displays the word “regrets” until the point that one looks directly at it, and switches to “regards”. The peripheral vision detects the discrepancy—the camera is really rather primitive and the pupil is often difficult to track in its 160x120 pixel output. The effect, I’m told, is disquieting.
There is a further significance to the black dialogue box; we might compare it with the dialogue cards in early silent films. In those films, characters are shown speaking (here we do in fact usually see their mouths moving, or at least we see their gesticulations) and the footage cuts to a black screen with white dialogue that tells us what has been said. There is a peculiar sense of suspense here; we see people speaking and only once they have finished are we allowed to know what it was they said. The cut between footage and dialogue is natural enough. The language of film was always destined to use the method of teleporting our field of view instantly between cuts and the hierarchy of the seen over the heard to determine that we must experience the dialogue in disjunction, in this order, and in this way. But what if it had been otherwise? What if the films had handled dialogue not by shredding the flow of linear time but instead by shredding the space of the shot? That is, by bringing up a box that covered the lower third of the screen to display what was being said.
Michael Snow, understanding the fertility of the disjunction between screen and audience in a way that the earlier filmmakers understood the disjunction between times and spaces in the montage, once made a film called So Is This: a silent film entirely consisting of white text displayed one word at a time over black. That the words appear one at a time is important because the audience must read at the exact same pace. In what I regard as the climax of the film the audience are asked to read the words for Somewhere Over the Rainbow, together, and in total silence. Snow describes this as an “optical cranial sing-song”. The music rings in each head, and the loneliness of the private mind is for a short while abolished. When two people sit together and play Mother 3 on the Nintendo DS, the gaze tracker doesn’t know what to do. It looks for pupils, finds too many, and rapidly flickers between every possible point of view between each hypothetical pair of eyes. The text cannot now be disquieting: it looks like a man hiding behind a single tree from a crowd that surrounds him on all sides. It looks silly, and then we remember we’re talking to a frog in a video game.
The Shining Ice
Ballet for Inanimate Dancers
Eyes and Projection