How a 64k intro is made (2017)

back to lofibucket.com

An intro to intros

The demoscene is set producing chilly trusty time works (as in “runs for your computer”) known as demos. Some demos are in point of fact exiguous, explain 64 kilobytes or less, and these are known as intros. The name comes from “crack intros”. So an intro is correct a demo that’s exiguous.

I’ve observed many of us contain curiosity in demoscene productions however plan no longer contain any idea how they’re in point of fact made. Right here’s a braindump/autopsy of our recent 64k intro Guberniya and I am hoping that this could possibly be inviting to beginners and seasoned veterans alike. This text touches most continuously all ways worn within the demo and could possibly mild provide you with an idea what goes into making one. I consult with folks with their prick names listed right here on myth of that’s what sceners discontinue.

Guberniya in a nutshell

Windows binary download: guberniya_final.zip (61.8 kB) (critically broken on AMD playing cards)

It’s a 64k intro released at Revision 2017 demo occasion. Some numbers:

• C++ and OpenGL, dear imgui for GUI
• 62976 byte Windows executable, packed with kkrunchy
• largely raymarching
• 6 person team
  • one artist 🙂
• in-constructed four months
• ~8300 traces of C++, library code and whitespace excluded
• 4840 traces of GLSL shaders
• ~350 git commits

Development

Demos are most continuously released at a demo occasion where the audience watches demos submitted to a contest after which votes for the winner. Releasing at a occasion is a mammoth strategy to derive motivated on myth of you could possibly want a difficult deadline and an fervent audience. In our case it modified into Revision 2017, a large feeble demo occasion at some point soon of the Easter weekend. It is probably going you’ll possibly survey some photography to derive an idea what the tournament is like.

We started working on the demo early in January and released it on the Easter weekend in April at some point soon of the occasion. It is probably going you’ll possibly stare a recording of the entire competition online whenever you occur to want 🙂

We were a team of six: cce (me), varko, noby, branch, msqrt, and goatman.

Make & influences

The tune modified into executed correct early on, so I tried to create issues spherical it. It modified into determined we wanted one thing large and cinematic with memorable situation pieces.

My customary visual suggestions centered spherical wires and their usage. I in point of fact loved Viktor Antonov’s designs and my first sketches were correct much a rip-off of Half-Existence 2:

The similarities are moderately evident. In the panorama scenes I modified into furthermore making an strive to steal the mood of Eldion Passageway by Anthony Scimes.

The panorama modified into inspired by this advantageous video of Iceland and furthermore Koyaanisqatsi, I dispute. I furthermore had large plans for the memoir that manifested itself as a storyboard:

If I’d discontinue this again I’d correct resolve with a timeline with a pair of photography that situation the mood. It’s less work and leaves extra room for creativeness. But on the least drawing it compelled me to put together my thoughts.

The Ship

The spaceship modified into designed by noby. It is a aggregate of multiple Mandelbox fractals intersected with geometric primitives. The ship’s create modified into left a bit of incomplete, however we felt it shouldn’t be extra tampered with within the closing model.

We had furthermore one other ship shader that didn’t derive worn. Now that I look on the create it’s furthermore very chilly and it’s a disgrace it didn’t catch expend within the intro.

Implementation

We started with a codebase constructed for our earlier intro Pheromone (YouTube). It had total windowing and OpenGL boilerplate on the side of file gadget utility that packed files from an data directory to executable with bin2h.

The workflow

We worn Visible Studio 2013 to assemble the project because it wouldn’t assemble on VS2015. Our customary library replacement didn’t work effectively with the as much as this point compiler and produced fun errors like this:

For some reason we stuck with VS2015 as an editor even though and correct compiled the project utilizing the v120 platform toolkit.

We had a straightforward global keyboard hook that reloaded all shaders when CTRL+S key aggregate modified into detected:

// Hear to CTRL+S.
if (GetAsyncKeyState(VK_CONTROL) && GetAsyncKeyState('S'))
{
    // Await a while to let the file gadget create the file write.
    if (system_get_millis() - last_load> 200) {
        Sleep(100);
        reloadShaders();
    }
    last_load=system_get_millis();
}

This worked in point of fact effectively and made live editing shaders much extra fun. No want to contain file gadget hooks or the relaxation.

GNU Rocket

For animation and route we worn a GNU Rocket fork Ground Regulate. Rocket is a program for editing animation curves and it connects to the demo by a TCP socket. The keyframes are despatched over when requested by the demo. It’s very convenient on myth of you could possibly edit and recompile the demo while conserving the editor open with out dropping the sync relate. For the closing release the keyframes are exported to a binary structure. It has some demanding limitations even though.

The Tool

Transferring the perspective with mouse and keyboard is extremely helpful for selecting camera angles. Even a straightforward GUI helps loads when tweaking values.

We didn’t contain a demotool unlike some folks so we needed to make it as we went an extended. The top likely dear imgui library allowed us to effortlessly add aspects as we wanted them.

As an instance including some sliders to construct an eye on some bloom parameters is so simple as including these traces within the rendering loop (no longer to separate GUI code):

    imgui::Originate("Postprocessing");
    imgui::SliderFloat("Bloom blur", &postproc_bloom_blur_steps, 1, 5);
    imgui::SliderFloat("Luminance", &postproc_luminance, 0.0, 1.0, "%.3f", 1.0);
    imgui::SliderFloat("Threshold", &postproc_threshold, 0.0, 1.0, "%.3f", 3.0);
    imgui::Discontinue();

The pause result:

The camera relate will more than likely be saved by pressing F6 to a .cpp file, so the next time the code is compiled this could possibly be included. This avoids the need for a separate data structure and the associated serialization code, however this solution can furthermore derive correct messy.

Making exiguous binaries

The main to exiguous executables is scrapping the default customary library and compressing the compiled binary. We worn Mike_V’s Little C Runtime Library as a deplorable for our contain library implementation.

The binaries are compressed with kkrunchy, which is a instrument made for precisely this reason. It operates on standalone executables so you could possibly write your demo in C++, Rust, Object Pascal or irrespective of. To be legal, dimension wasn’t in point of fact a controversy for us. We didn’t retailer much binary data like photography so we had deal of room to play with. We didn’t even expend away comments from shaders!

Floating aspects

Floating point code precipitated some complications by producing calls to nonexistent customary library functions. All these were eradicated by disabling SSE vectorization with the /arch:IA32 compiler switch and taking away calls to ftol with the /QIfst flag that generates code that doesn’t place the FPU truncation mode flags. Right here is no longer a controversy on myth of you could possibly situation the floating point truncation mode first and main up of your program with this snippet courtesy of Peter Schoffhauzer:

// situation rounding mode to truncate
//  from http://www.musicdsp.org/showone.php?identification=246
static short control_word;
static short control_word2;
inline void SetFloatingPointRoundingToTruncate()
{
    __asm
    {
        fstcw   control_word                // retailer fpu build an eye on notice
        mov     dx, notice ptr [control_word]
        or      dx, 0x0C00                  // rounding: truncate
        mov     control_word2, dx
        fldcw   control_word2               // load modfied build an eye on notice
    }
}

It is probably going you’ll possibly read extra about these items at benshoof.org.

POW

Calling pow mild generated a call to __CIpow intrinsic characteristic that didn’t exist. I couldn’t determine its signature on my contain however I came all the draw by an implementation in Wine’s ntdll.dll that printed that it expects two double precision floats in registers. Now it modified into that you just could possibly imagine to create a wrapper that calls our contain pow implementation:

double __cdecl _CIpow(void) {
    // Load the values from registers to local variables.
    double b, p;
    __asm {
        fstp qword ptr p
        fstp qword ptr b
    }
    // Implementation: http://www.mindspring.com/~pfilandr/C/fs_math/fs_math.c
    return fs_pow(b, p);
}

While you understand a nicer strategy to fix this, please let me know.

WinAPI

While you could possibly’t depend on SDL or an identical it be vital to make expend of terrifying WinAPI to discontinue the dear plumbing to derive a window on cloak cloak. While you could possibly be suffering by this, these could possibly grunt purposeful:

• WinAPI window creation example
• OpenGL initialization example, requires glext.h and wglext.h

Demonstrate that we completely load the characteristic pointers for OpenGL functions that are in point of fact worn within the production within the latter example. It goes to be a legal recommendation to automate this. The functions could possibly mild be queried with string identifiers that derive kept within the executable, so loading as few functions as that you just could possibly imagine saves situation. Entire Program Optimization could possibly derive rid of all unreferenced string literals however we couldn’t expend it thanks to a controversy with memcpy.

Rendering ways

Rendering is basically raymarching and we worn the hg_sdf library for convenience. Íñigo Quílez (to any extent extra known as correct iq) has written a lot about this and many of the ways. While you’ve ever visited ShaderToy you wish to be acquainted with this already.

Additionally, we had the raymarcher output a depth buffer rate so lets intersect signed distance fields with rasterized geometry and furthermore notice publish-processing effects.

Shading

We expend customary Unreal Engine 4 shading (right here’s a large pdf that explains it) with a GGX lobe. It isn’t very considered however makes a distinction in highlights. Early on our thought modified into to contain an unified lighting fixtures pipeline for both raymarched and rasterized shapes. The muse modified into to make expend of deferred rendering and shadow maps, however this didn’t work at all.

Rendering large terrains with shadow maps is mammoth hard to derive correct thanks to the wildly varying cloak cloak-to-shadow-scheme-texel ratio and other accuracy complications. I wasn’t in point of fact within the mood to originate up experimenting with cascaded shadow maps either. Also, raymarching the identical scene from multiple aspects of survey is slack. So we correct determined to scrap the entire unified lighting fixtures part. This proved to be a massive effort later when were making an strive to match the lighting fixtures of the rasterized wires and raymarched scene geometry.

Terrain

The terrain is raymarched rate noise with analytic derivatives.¹ The generated derivates are worn for shading of route, however furthermore to construct an eye on ray stepping dimension to bustle ray traversal on gentle regions, correct like in iq’s examples. While you like to have to study extra you you could possibly read extra about this technique in this dilapidated article of his or mess spherical alongside with his correct rainforest scene on ShaderToy. The panorama heightmap became much extra realistic after msqrt implemented exponentially distributed noise.

The panorama discontinue is extremely slack on myth of we discontinue brute pressure shadows and reflections. The shadows expend a soft shadow hack in which the penumbra dimension is situation by the closest distance encountered at some point soon of shadow ray traversal. They give the impression of being correct advantageous in movement. We furthermore tried utilizing bisection tracing to high-tail it up however it completely produced too many artifacts to be edifying. Mercury’s (one other demogroup) raymarching recommendations on the opposite hand helped us to eke out some extra quality with the identical velocity.

The sky is constructed utilizing correct much the identical ways as described by iq in on the back of elevated, trek 43. Correct some simple functions of the ray route vector. The solar outputs correct mammoth values to the framebuffer (>100) so it provides some natural bloom as effectively.

The alley scene

Right here’s a survey that modified into inspired by Fan Ho’s photography. Our publish-processing effects in point of fact create it attain together even though the underlying geometry is correct simple.

Modelling with distance fields

The B-52 bombers are a legal example of modelling with signed distance fields. They were much extra good within the occasion model, however we spiced ’em up for the closing. They give the impression of being correct convincing from afar:

On the opposite hand they’re correct a bunch of capsules. Admittedly it can possibly’ve been more uncomplicated to correct to create them in some 3D bundle however we didn’t contain any roughly mesh packing pipeline situation up so this modified into sooner. Correct for reference, right here is how the space enviornment shader appears as if: bomber_sdf.glsl

The characters

The racy characters are correct packed 1-bit bitmaps. At some stage in playback the frames are crossfaded from one to the next. They were contributed by a mysterious goatman.

Submit-processing

The publish-processing effects were written by varko. The pipeline is:

1. Apply shading from G-buffer
2. Calculate depth-of-enviornment
3. Extract bright parts for bloom
4. Compose N separable Gaussian blurs
5. Calculate fraudulent lens flares & large headlight flares
6. Composite all together
7. Comfy edges with FXAA (thanks mudlord!)
8. Shade correction
9. Gamma correction and refined film grain

The lens flares apply correct much the technique described by John Chapman. They were most continuously hard to work with however within the pause mild delivered.

The depth of enviornment discontinue (per DICE’s technique) is made of three passes. Essentially the dear one calculates the dimensions of circle of bewilderment for every pixel and the 2 other passes notice two rotated box blurs every. We furthermore discontinue iterative refinement (i.e. notice multiple Gaussian blurs) when wanted. This implementation worked in point of fact effectively for us and modified into fun to play with.

The depth of enviornment discontinue in movement. The pink notify reveals the calculated circle of bewilderment for the DOF blur.

Shade correction

There could be an racy parameter pp_index in Rocket that is worn to replace between color correction profiles. Every profile is correct a good branch in a large switch assertion within the closing publish-processing movement shader:

vec3 cl=getFinalColor();
if (u_GradeId==1) {
    cl.gb *=UV.y 0.7;
    cl=pow(cl, vec3(1.1));
} else if (u_GradeId==2) {
    cl.gb *=UV.y 0.6;
    cl.g=0.0+0.6*smoothstep(-0.05,0.9,cl.g*2.0);
    cl=0.005+pow(cl, vec3(1.2))*1.5;
} /etc.. */

It’s very simple however worked effectively sufficient.

Physics simulation

There are two simulated programs within the demo: the wires and a flock. They were furthermore written by varko.

The wires

The wires are truly appropriate as a chain of springs. They’re simulated on the GPU utilizing compute shaders. We high-tail multiple exiguous steps of the simulation because of the the instability of the Verlet integration draw we expend. The compute shader furthermore outputs the wire geometry (a chain of triangular prisms) to a vertex buffer. Sadly, the simulation doesn’t work on AMD playing cards for some reason.

A flock of birds

The birds give a sense of scale.

The flock simulation includes 512 birds with the first 128 truly appropriate because the leaders. The leaders switch in a curl noise sample and the others apply. I believe in trusty lifestyles birds expend into myth the movement of their closest neighbours, however this simplification appears legal sufficient. The flock is rendered as GL_POINTs whose dimension is modulated to give appearance of flapping wings. This rendering technique modified into furthermore worn in Half-Existence 2, I believe.

Song

The feeble strategy to create song for a 64k intro is to contain a VST-instrument plugin that enables a musicians to make expend of their traditional instruments to create the song. Farbrausch’s V2 synthesizer is a classic example of this draw.

This modified into a controversy. I didn’t want to make expend of any prepared made synthesizer however I furthermore knew from earlier failed experiments that making my contain virtual instrument could possibly be loads work. I consider in point of fact liking the mood of ingredient/gesture 61%, a demo by branch with a paulstretched ambient tune. It got me implementing it in a 4k or 64k dimension.

Paulstretch

Paulstretch is a stupendous algorithm for in point of fact crazy time stretching. While you haven’t heard about it, you could possibly mild positively pay attention what it will create out of Windows 98’s startup sound. Its internal workings are described on this interview with the writer, and it’s furthermore open supply.

Most continuously, as it stretches the enter it furthermore scrambles its phases in frequency situation in grunt that in preference to metallic artifacts you derive ethereal echoes. This requires of route a Fourier change into and the contemporary application uses the Kiss FFT library for this. I didn’t want to depend on an external library so within the pause I implemented a naive O(N²) Discrete Fourier Change into on the GPU. This took a truly long time to derive correct however within the pause it modified into worth it. The GLSL shader implementation is extremely compact and runs correct hasty despite its brute-pressure nature.

A tracker module

Now it modified into that you just could possibly imagine to create swathes of ambient drone, given some cheap enter audio to stretch. So I sure to make expend of some tried and tested technology: tracker song. It’s correct very reminiscent of MIDI² however with furthermore samples packed within the file. As an instance elitegroup’s kasparov (YouTube) uses a module with extra reverb added. If it worked 17 years ago, why no longer now?

I worn Windows’ constructed-in gm.dls MIDI soundbank file (again, a classic trick) to create a tune with MilkyTracker in XM module structure. Right here is the structure that modified into worn furthermore for many MS-DOS demoscene productions back within the 90s.

The take care of with gm.dls is that the devices, courtesy of Roland in 1996, sound very dated and cheesy. Turns out right here is no longer a controversy whenever you occur to bathe them in a full bunch reverb! Right here’s an example where a temporary test tune is performed first and a stretched model follows:

Surprisingly atmospheric, correct? So yeah, I made a tune imitating Hollywood songwriting and it became out mammoth. That’s correct much all that’s occurring the song aspect.

Thanks

Thanks to varko for back in some technical vital aspects of this text.

• Ferris of Logicoma exhibiting off his 64k toolkit
  • Get sure to first stare Prefer, their contestant within the identical compo we took section in
• The supply of some Ctrl-Alt-Test productions
  • Has some 4k and 64k code.
  • That they had an intro too: H-Immersion