My quest in procedural content.

Posted by Gordy on April 12, 2012, 8:18 a.m.

Introduction

Be forewarned, this is a wall of text.

Most of the things I made was because I was trying to solve an issue I had with my current game engine. Most of these things will be released with the tLibrary, a set of scripts and mathematical functions for advanced GM users.

Data structures & Algorithms

tQuery

Originally, while writing a game saving system for Kingsfield, even after playing for just 5 minutes, the save file consisted of over 32 chunk files, 1 file for the player, 1 file for nation, country, and town data, and 2 files for NPC + Lore data. All the files combined, it was ~30mb, and would probably grow to double that throughout the course of regular gameplay. The load time was atrocious, and I was not happy with needing so many files.

I then realized I needed a new method for saving, so, I looked into MySQL.

Now, there are DLL's for using MySQL with Game Maker, but I didn't want to require the user to have an internet connection to save their game. SQLite was an option, but I figured, what the hell, might as well try to write my own flat file storage directly with GM.

I do know that using a DLL would be faster, more efficient, and since I've been using MySQL for a while, wouldn't be hard for me to use. But, I like the idea of keeping things self contained, and using as minimal extensions as possible.

Call me a crazy, but I love busting my brain to try and make new technologies with pure Game Maker.

There's not many visuals, as you can probably expect, but I ran some benchmarks against the SQLite DLL to see how it compared, and these are the results, using Yourself's highResTimer.dll:

Code for SQLite:

timeLast = external_call(timerDLL  ,  0  ,  1000000);
query(db,"CREATE TABLE t1(a INTEGER PRIMARY KEY, b VARCHAR(62))")
repeat (10){
query(db,"INSERT INTO t1 VALUES(NULL,'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789')")
}
show_message(string(external_call(timerDLL  ,  0  ,  1000000)-timeLast));

Code for tQuery:

timeLast = external_call(timerDLL  ,  0  ,  1000000);
tquery("t1",ADDCOLUMN,"text");
repeat (10){
tquery("t1",ADD,"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789");
}
tq_push();
show_message(string(external_call(timerDLL  ,  0  ,  1000000)-timeLast));

You'll notice a quarter of the way through, I stopped benchmarking the DLL, because at this point I figured something was wrong. 10 seconds for 100 inserts is absurd, and so I went to find another DLL.

To my dismay, it also took 10 seconds to insert the 100 rows. I was wrong in thinking that a SQLite DLL would be an efficient way to save data.

But I continued on in my benchmarking, and was very surprised by the results. I was almost definitely sure it would be exponentially slower than any DLL.

I would call this a success.

QuadTrees

Of course, like many useful algorithms, there is no GM equivalent, and usually little documentation of GM users ever using/making them.

There were a few quadtree-like implementations however, including a very decent one by xot, who I give credit to for inspiring my code.

As you can see, I chose Manhattan distance over Euclidean, for the sake of computation speed.

Runs at about 11 fps, that is when checking the cells every step, but I've gotten it up to 200+ when updating the cells dynamically based on their size, and at different intervals.

Both points and lines can be returned, depending on your needs.

Noise

Noise, is a great thing, but no so much by itself. Without some way to interpolate it, there are very few things you can do it.

At first, I figured it would take too much time generate a noise algorithm with even a percent of the quality Perlin Noise has, and I was mostly right. It's most certainly possible, but not with Game Maker. However, using Perlin's Simplex Noise as a model, I was able to create something halfway decent, although, still many times less efficient.

Here was my process:

1. Figured linear interpolation between two points was sufficient. Was not.

My first attempt failed, and the second, even after multiple iterations it still left noticeable artifacts. (and was extremely slow, about 30 seconds to render)

Second method, while testing some attenuation methods, I did however create a neat effect.

2. Linear interpolation would not produce adequate results, so I opted for another method, bicubic interpolation. (of course, I had to write my own bicubic interpolation script, because it was absolutely non-existent in anywhere in the GM realms)

This proved to be massively slow, requiring nearly one full minute to render a 512^2 image.

It did however produce a nice image.

a low resolution render

3. Gave up. There's a reason Perlin Noise is so famous, it works, and it's fast.

Voronoi Diagrams

I've had an interest in Voronoi Diagrams for a while, they proved very helpful for a few things I was working on.

While trying to figure out to how get them to work, I was plotting the results, here are some images while testing an attenuation function.

Currently it doesn't return the vertexes where three plots meet, but it shouldn't be hard to add, as I'm pretty sure it's just a function of the positions.

Delaunay Triangulation

I originally wanted to use delaunay triangulation to fix the holes that were appearing in my 3D terrain when changing detail levels. Ultimately I came up with a different solution to that, but only after I made this. Could be used for pathfinding or something, maybe generating hulls around a point cloud.

This was actually a lot more difficult than I anticipated, and alas, there was little information about it, practically none when it came to doing it in GM.

I ended up using priority lists, (first time ever using those, quite helpful) but, it's still quite slow at the moment. I haven't gotten around to optimizing it, and probably won't unless I choose to include it in tLibrary. (takes around 2 seconds to generate with 200 nodes)

Procedural Generation (Terrain + Civilzation)

I love the idea that we can use seemingly random data to generate coherent structures and data. So, that's what I set out to do.

Terrain

I first set out using a Perlin Noise DLL, which gave me great results.

i used voronoi diagrams to set the color of the terrain

by recursively splitting the voronoi plots, i was able to get a quite detailed color map

by using perlin noise twice, i was able to set heights not based on distance to water

After a while of playing with Perlin Noise, I once again, set out to do it without using a DLL.

This time, I used my Voronoi Diagrams as a base.

These turned out very nicely, and because I'm not using Perlin Noise, i'm able to more easily change the map generation parameters.

The next thing I did was divide the world into nations, then countries within those nations. Nation data includes, when it was formed, it's alignment, the conflicts and interactions with other nations, nation race makeup, nation treasures, and leaders; same goes with countries.

NPC's

Worked on an algorithm that generates names, stats, and recent history of a character based on their race, and occupation.

Also, when history is added to them, the corresponding history is also added to the other parties, not limited to NPCs, but also countries, towns, and nations.

Works nicely, and I plan on adding more data to NPCs.

Lore & History

Much like NPCs, history and stories are added to the world lore throughout the game.

I'm currently working on a "story generator" which takes data from the guilds, nations, npcs, countries and generates a story based on them. It also works in reverse, the more books you read in-game, the more detailed history will become.

Hopefully, this will encourage players who want to know more about something, to search and explore for the right book, or NPC to enlighten them.

Stories NPCs will tell are a bit different, I'm working on a system to inject data into partially prebuilt story templates, it looks something like this:

(H = Hook, M = Main Thesis, D = Details, T = Transition)

Story setup (H>M>M>D>M>D>D>D>T)

Story conflict (H>M>D>M>D>D>M>D>D>D>M>M>D>T)

Story climax (H>M>H>M>D>D>D>H>M>M>D>D>T)

Story resolution (M>H>M>M>M>M>D>M>H>T)

It's a bit rough, I'm having my writer work on improving it.

3D Developments

LOD / GeoMipMapping

My hope, is that you should be able to just barely see the Moutains of Elgamar from the Isles of Turdain, even if they're across the map. This meant I couldn't have a ton of geometry making up my model, because nearly all of it would be drawn.

So, I decided to make terrain LOD. Of course, little could be found about doing this in game maker, so I started coding.

At first, I split the terrain into chunks, and drew more detailed chunks closer to the player. This however left holes in the terrain where two different resolution chunks met. No good.

So I came up with a new solution, which is where my QuadTree came in handy.

Essentially, what I'm doing, is modifying the vertex height to an interpolated value of an adjacent chunks vertexes, if they're not of the same resolution.

This works, quite well, and I'm very happy with it.

3D Collisions

I could have just used the heightmap for terrain collisions, but then I'd have to come up with another system for other solid object collisions.

So, I spent a good day reading up on 3d triangle intersections, and working out how to do it, with yet another time, pure GML.

really lame screenshot, i know

So far, the results are looking really promising, smooth collisions, and rolling down inclines based on their normals, but I've yet to do any benchmarking.

Game Developments

Ancient

Still mostly the same as it was before, although I've finished all the levels, and am now waiting on the Musician to finish his work. Once he's done, Ancient will be released!

No new screenshots for this.

Roguelike

Just the start to a short game idea I had. It is no longer in development because Kingsfield is Roguelike with more content.

Kingsfield

Inspired by two of my most favorite games, D&D and Rogue, with elements from Dwarf Fortress, Fable, Shadow of the Colossus, Dragon Quest, Kingsfield, Kingdom Hearts, Radiata Stories + Eternal Sonata, Diablo, and the Elder Scrolls series.

It's essentially a procedurally generated rpg, strongly based on exploration.

With an active Lore system, NPC's, Landmarks, History, Quests + Stories, are all generated as you play, so you will quite possibly never run out of things to do. (Using the Procedural techniques mentioned earlier)

No screenshots yet, since currently, i'm still working out the core engine.

you would not imagine how long this took to write..

Comments

JuurianChi 12 years ago

Quote:
you would not imagine how long this took to write..
In that case, I'll read it twice.

Taizen Chisou 12 years ago

I'm not sure what I'm looking at, but it sure is pretty sexy.

Gordy 12 years ago

@juurianChi

like, i'm pretty sure I clicked "Post Blog" at 4:00am; the sun has been up for a while now..

Toast 12 years ago

Hey

Hey

Enter the competition

Please

But seriously, this was an interesting read, particularly the stuff on noise and procedural generation as I'll be doing similar stuff soon.

sirxemic 12 years ago

Nice read, I do have a few questions though.

Quote:
As you can see, I chose Manhattan distance over Euclidean, for the sake of computation speed.
No, I cannot see that. Can you explain it in a bit more detail? I have worked a bit with quadtrees myself and I cannot figure out what you are talking about.

Quote:
That is awesome.

What is that picture under "Delaunay Triangulation" supposed to show?

Your roguelike game reminded me of the fact how that graphical style is starting to lose its originality - http://www.youtube.com/watch?v=dattsmkfzhU

JuurianChi 12 years ago

Yeah, over saturated market is over saturated.

Gordy 12 years ago

@sir xemix

i just ran some benchmarks, repeated the code 128^2 (the smallest cell size I have).

point_distance(x1,y1,x2,y2) vs (x2-x1)+(y2-y1)

euclidean distance took 27472(0.027s) and manhattan took 21437(0.021s).

it seems i was wrong thinking that it would be comparably slower, i may go back to using it.

delaunay trianglulation connects a vertex all its neighbors without crossing the path of another connection. (i used a distance falloff, which is why it's not completely triangulated). other than that, it doesn't show much.

Acid 12 years ago

This is one of the most impressive things I've read in a while.

LAR Games 12 years ago

Mhmm. Yeah… I know some of these words.

Praying Mantis 12 years ago

You're going to go far.