The world is [fill in here]: experiment on a pluralistic world

February 2024

World shapes map, map shapes worlds

Imagine you are a Polynesian islander living on the Marshall Islands a couple hundred years ago, and 97.87% of your territory is made out of water. The result of your circumstances is that all of your maps look like this:

Stick map

This is a Micronesian stick chart and it shows the wave and wind patterns of the surrounding ocean. If you have just become a Marshallese clan member after being dropped down from a time machine previously set to the year 2024, chances are, you have no idea what the sticks symbolize and what their placement in space refers to. Living in the world with no spatial referencing appears real tricky, real suddenly. To a regular old Marshallese though, this kind of map clearly represented the most important features of their living space: the relevant reference points to orient themselves and just go about their business in a world full of water and waves.

Your experience of maps is probably different, made of globes and graticules. Let’s take a quick look at how that came about, the simplified evolution of cartography, or the art of map making:

1. Pre-modern era: You make maps of your immediate surroundings (local representation).

   The oldest surviving map (25,000 BC) - shows a mountain, river, valleys and routes on a mammoth tusk

2. Expedition period: you travel the world (global representation). From Juan de la Cosa to Alexander von Humboldt, you push the horizons of what was previously known. Congrats, you now have a representation of the world to a first approximation.
3. Modern times: you increase accuracy and capture all kinds of geographic details (topographical, bathymetric, roads...), match the metaphorical details to the updated zeitgeist (borders, constellations...), sometimes sprinkle in political and economic information, remove errors (except the ones you put there on purpose), and put it all on the internet.
4. Now: here we are, you live in the world of today, in which you take all of the earthly maps of the world with you in your pocket, regardless of whether you are heading to a supermarket or the Tibetan Plateau. What now? Is cartography over? No more here be dragons, long gone the terra incognita? Should we repeat steps 2 and 3, but for the entire universe? That seems like a pretty cool idea, but probably requires making some outer space tech advances first.

This is it then! We are the people of an anticipatory generation, stretched between the nostalgia of youtube videos of Buzz Aldrin stomping on the Moon, images from the James Webb Space Telescope, and the hope of SpaceX’s Starships disintegrating a split second after reaching orbit, rather than before.

Hold your explorer's tears back for a second and do not despair just yet, there might be another direction worth looking into. With @mattlecauchois and @aurelien_morel we’ve been devising a perceptual atlas, so let’s dive into what we mean by that.

Maps aren’t just a visual depiction of an area: they are a reference frame for possible and allowed actions. As per cartographer Denis Wood, maps are an argument about the world. Nowadays, we all use roughly the same world map, and its main function is optimizing for the shortest travel time between point A and point B. You could technically optimize for other travel measures, like number of scenic spots, sunshine exposure, or more generally, the amount of qualia available. But maybe, as maps are still getting more features built in, the next use of maps isn’t solely for traveling.

A relatively new feature of maps is Google Street View. Google performed the silly little task of putting a car on ~every street in the world to photograph it. For perspective, that is more than 220 billion images spanning 10 million miles over 100 countries. Street View, unlike classic maps depicting the world from a bird’s or God’s viewpoint, is a 1:1 map of the world from your perspective. There is no Mercator projection, no distortion - it’s the raw, first-person experience, minus a few blurry faces and license plates. Google could rebrand as Borges’s Cartographers Guilds who “struck a Map of the Empire whose size was that of the Empire, and which coincided point for point with it.” Now, speaking from experience, I think we have been using the Google app in very limited ways (checking out your own house, or enjoying some Geoguessr tours de force). Street View is currently underrated, and maybe a more interactive or controllable or immersive version is needed to make full use of its potential.

Thinking about the evolution of maps into dynamic, interactive tools, we have witnessed engaging geospatial innovations like Pokemon Go, where a unique virtual world is seamlessly integrated with our real-world surroundings. The major boundary of the Niantic Labs tech is that, in its current form, it is not allowing anyone to change the world based on their own input.

The case of the vibes, and vibes-based worldmaking

As a human, I want to enable more creative arguments (the Dennis Wood kind mentioned above), let people express it visually (or via other high bandwidth medium), and show their argument to themselves and others. This reflective phenomenology is something that new technologies like AI image generation bring as a whole new capability: we can now superimpose our seeing the world in a certain way and imagining what the world can be.  With this, we can greatly shorten the feedback loop between our inner world and a representation of it existing in the outer world; more than that! We can fine tune and share it with other people. It is such a satisfying way to get things out of your head.

This quick iteration loop is important because the truth is that, from what you’re experiencing right now, to what you’re projecting out into the world, it’s all vibes, which can sometimes be hard to define. What has been missing in the interactive vibes realm, what AI enables, is what I call a vibe knob. With a vibe knob, you can make the world match your vibe, you can even translate a vague feeling into a tangible inspiration - what would you like it to feel like?  Are you experiencing the same reality as I am? Who knows, here’s what it feels like for me, here’s what would make me feel even better. The goal is to provide a description of what you experience in perceptual terms. Then, you can exchange it with others, feel what they perceive, and through this process of engaging with the world, you change it.

VLADIMIR: Well? Shall we go?

ESTRAGON: Yes, let’s go.

[They do not move.]

Waiting for Godot

If you’re a metaphysical realist or you think there are only subjective realities, you won’t mind distorting it just a little more, maybe only to summon all the possible worlds. AI makes this a lot easier, while giving you more capabilities to 10x what you had in mind. A good example is the visual essays in the first issue of telescope magazine, which activates a lot of my neurons. It’s about creating compelling, beautiful narratives, and I just haven’t seen things like this before.

In the attempt to fabricate, to bring into existence, to conjure up from the depths of the void something akin to the New, we find ourselves hemmed in on all sides by the immutable laws of creation itself. These are not the physical laws, not the kind that govern the motion of celestial bodies or the behavior of particles at the quantum level, but rather the metaphysical, the epistemological constraints that bind us to the continuum of what has come before. Everything is a remix. We can't just pull rabbits out of a hat, everything has roots and we follow what came before. However, we can affect the creative product and bring it to the of the scene. Some examples of what I am talking about:

Picasso’s Las Meninas, modified from Velazquez

Picasso’s is a less realistic representation in the commonly understood sense (which itself is less realistic than if he had taken a 35mm photo). I would argue, though, that Picasso’s representation of reality is just as valid as Velazquez’s. In worldmaking, you need to find the right balance between prediction error and familiarity. If you create worlds from scratch, it becomes pure exploration because of huge prediction errors, and is harder to make sense of. You can even start from existing world representations, like one can take impressionism and go even further in this iterative process:

Banksy’s Show me the Monet

As Nelson Goodman says in Ways of Worldmaking, it’s not about the truth; reality is relative and the physicist and the philosopher will disagree on what is there. We live in worlds of gods and democracies and poetry and work and relationships, and also there’s a moving limit to what we do know (force of gravitation but general relativity but quantum theory...)  It’s facts made of theories and theories made of facts. The whole truth is too much, the truth alone is too little, the vibes-based truth is just right.

So what is at the intersection of cartography and vibes-based worldmaking?

The head-to-world interface experiment

The v0 prototype is an app that filters the Google Street View through your own perceptual preference lens. You - the user - can increase symmetry or make a blueberry world, there are no rules to your worldmaking; however, that doesn’t mean you don’t have to think about what you’re doing. Per Goodman:

While readiness to recognize alternative worlds may be liberating, and suggestive of new avenues of exploration, a willingness to welcome all worlds builds none. Mere acknowledgement of the many available frames of reference provides us with no map of the motions of heavenly bodies; acceptance of the eligibility of alternative bases produces no scientific theory or philosophical system; awareness of varied ways of seeing paints no pictures. A broad mind is no substitute for hard work.

You can literally imagine the world you want to live in, and the question then becomes how to get from world A to world B. Actionable urban planning or simply new conversations, it’s worth a try.

We diffuse from Google Street View 360 panoramas. There are 2 modes, remix using ControlNet and morph using SDXL Turbo. Given the same prompt and location, the two modes have completely different feels.

original street view

In general, remix keeps more consistency while morph shoots up the prediction eror. To stitch each of the three 120° shots together we used basic inpainting with stable-diffusion-2-inpainting. It requires a bit of prompt and location engineering for the inpainting to work well, but some worlds are really neat. There’s an example and a share button to help with curation. You can use the arrow keys to turn around, and the space bar to see the aligned original shot.

Welcome to the perceptual atlas, the new cartography, a map of the heart.

This was available for a limited time (compute is expensive lol) at panoramai.xyz. Fun experiment, 10k people tried it out, it topped HackerNews and was featured on Ben's Bites.

You can run your own server using the source code available here.

Thanks to Martyna Stachaczyk for helping shape and sometimes express my thoughts in writing, and Matthieu Le Cauchois & Aurélien Morel for reading drafts of this.