CC65 memory optimization tips

The main engine for my arkanoidvania game is done. I’m now working on adding stuff. This is where the fun part begins, or so I thought! One day the compiler threw this error at me: “BANK1 overflows memory area ‘PRG1’ by 1byte.” Wait! What? Which byte should I delete? So I went on a journey to optimize my data and my code to free memory space so I can squeeze some awesome brick breaking levels in there.

I made some experimentation to try and tell the compiler to save some precious bytes. There is already a great article here on how to optimize the performance. It’s a must read. My article is just an humble add-on, from my experience. It is not meant to get a faster code, although a shorter code should be faster in theory. It is about saving space.

The RLE compression

RLE (or run-length encoding) is a simple data compression method, useful for compressing tilemaps. For my temple dungeon, I wanted to throw in some floor tiles to add textures. I went mad and added too much. I went back and decided to delete most of them. It looks more clean, and I saved 57 bytes!

Since RLE compresses a stream of data from left to right, it doesn’t change anything if we remove the bottom row (the little purple line). But on the sides, the compression must add a byte or two each time it encounters a new tile, so removing those tiles saves an extra 16 bytes. I could have removed more, but we must find the balance between adding decorative tiles and saving space. I went back to my other levels, removed a lot of useless decorative tiles, saved a ton of bytes and lived happily ever after.

The Nametable attributes can also takes extra bytes if you doesn’t set them right. For example, this level attributes are a mess. The whole nametable on the top takes 504 bytes, the one on the bottom 486. We saved 18 simply by setting our attributes right. There is a small color change on the grass tiles, but it doesn’t bother me. I now have to reopen and verify all my old levels to see if I messed up the attributes, this is so much fun!

Since I’m using 8×8 collision maps, it takes a lot of space. I coded my own compression which was similar to RLE, but I had to rethink it. I now compress line by line instead of a stream of data, and I take into account identical lines. I saved a lot. If you can make your own tool to save memory, it can worth the time and effort.

The importance of being constant

You must define and use constants as much as possible. I use a switch to load level specifics. Without thinking too much, I used my variable current_level to access my level array:

switch (current_level) {
    case LVL_TEMPLE1:
        ...
        if (levels.state[current_level] & LEVEL_EVENT1) {
            ...
        }
        ...
        break;
}

This code is in a context where I know exactly on which level I am. It is better to use the already defined constant LVL_TEMPLE1 instead of current_level:

        if (levels.state[LVL_TEMPLE1] & LEVEL_EVENT1) {
            ...
        }

It saves a little 2 bytes, but I used it often. I was able to save a lot.

oops, OOP hoops

One of the paradigm of Object Oriented Programming is to avoid code repetition at all cost. But C is not an OOP langage, let alone assembly. In this example, I need to setup 3 candle sprites when I load a specific level. To avoid repetition, I made a function to initialize them. Here’s the code:

void set_candle(unsigned char i, unsigned char x, unsigned char y) {
    actors.x[i] = x;
    actors.y[i] = y;
    actors.state[i] = IDLE;
    actors.animation_delay[i] = 8;
    actors.type[i] = TYPE_CANDLE;
}

// Later, in my load level code
set_candle(6, 32, 79);actors.current_frame[6] = 4;
set_candle(7, 40, 79);
set_candle(8, 216, 79);actors.current_frame[8] = 2;

This seems clever, at least for the OOP minded programmer, but it takes a lot of space! Maybe if I had a lot of candles to setup, it would be helpful, but I removed the function, and I set up the candles directly in the load level code. Notice that I changed the current_frame after calling the function. It was just so that the candles doesn’t flicker at the same time. Few people would even notice, but I like those small details! Although it doesn’t take much extra bytes, I decided to keep only one of the two assignation.

// Candles init
actors.x[6] = 32;
actors.y[6] = 79;
actors.state[6] = IDLE;
actors.type[6] = TYPE_CANDLE;
actors.current_frame[6] = 4;

actors.x[7] = 40;
actors.y[7] = 79;
actors.state[7] = IDLE;
actors.type[7] = TYPE_CANDLE;

actors.x[8] = 216;
actors.y[8] = 79;
actors.state[8] = IDLE;
actors.type[8] = TYPE_CANDLE;

Great, I saved 55 bytes! And it uses constant as array indexes. But we can save more by sorting our assignments by identical values:

// Candles init
actors.state[6] = IDLE;
actors.state[7] = IDLE;
actors.state[8] = IDLE;

actors.type[6] = TYPE_CANDLE;
actors.type[7] = TYPE_CANDLE;
actors.type[8] = TYPE_CANDLE;

actors.y[6] = 79;
actors.y[7] = 79;
actors.y[8] = 79;

actors.x[6] = 32;
actors.x[7] = 40;
actors.x[8] = 216;

actors.current_frame[6] = 4;

It made me save another 17 bytes, for a total of 72. It’s great to save those precious bytes, but the code is not very elegant and harder to maintain. If it didn’t save that much bytes, I wouldn’t have changed it.

There’s a little tool made by Shiru to evaluate how much space you take. It gives a nice overview. If you really want to know how much space you have left, just add an array filled with garbage bytes and the compiler will tell you.

Let me know in the comment section if this article was helpful, if it contains error or if you have other tricks. Now go and sin no more.