Notes of studying audio and images
COLOR AND GAMMA
Your eyes see brightness on a curve not a straight line. sRGB encodes images along that curve so 256 values feel evenly spaced. But if you do math on pixel values like blending or lighting you need to convert to linear space first or you get wrong results.
BIT DEPTH
Work in 16 or 32 bit internally. Only convert to 8 bit at the final export. If your pipeline drops to 8 bit too early you get banding and lose information you cannot recover.
IMAGE FORMATS
PNG for lossless. WebP for smaller files at same quality. Never use lossy compression on pixel art.
TEXTURE ATLASING
Pack many sprites into one image with a JSON coordinate map. One GPU texture bind instead of hundreds.
PNG METADATA
Use tEXt chunks not EXIF for storing data in PNGs. Things like frame count, collider type, collider vertices.
ALPHA PREMULTIPLICATION
If you see mysterious dark outlines around sprites check whether your pipeline expects premultiplied alpha but your images have straight alpha or vice versa. This is a common invisible bug.
PALETTE SWAPPING
Store one sprite. Swap the color palette at runtime for variants. Red knight blue knight green knight from one asset. This could be really useful for your modify system instead of regenerating entire sprites.
CONTENT ADDRESSABLE STORAGE
Hash file contents. Use the hash in the URL. Set cache to immutable. Content changes mean hash changes mean new URL. No cache invalidation problems.
AUDIO FORMATS
MP3 is fine for one shot voice lines and sound effects which is what you get from ElevenLabs. Your current pipeline of MP3 with silence trimming is good for that use case. If you ever need seamless looping audio like music or ambience switch to OGG or OPUS and request WAV from the source to avoid lossy to lossy conversion.
AUDIO FOR WEB
Browsers require a user click before audio can play. You already handle this. Keep short effects fully loaded in memory for instant playback. Stream longer music from the server in chunks so you do not eat RAM.
NETWORK STUFF
Gzip and brotli are useless on already compressed files like PNG and MP3. But they are huge savings on your JSON metadata. For real time audio like voice chat use UDP not TCP because a late packet is useless. Base64 inline images under 2 kilobytes. URL for anything bigger.
