Engineering Logs - VFS

Posted by Astryl on Jan. 13, 2015, 5:57 a.m.

It's a quiet day at work and I feel like doing something with my time.

So here's the second in a potentially long-lived series of blogs about what I'm doing with my engine.

Just a note, before I start:

These may or may not be the 'best' ways of doing things, but I really don't give a damn. I'll do things as I please, since I'm the one who has to work with the end product.

Everybody is welcome to work in whatever way they think is best. This is my way of working.

VFS

In Exile I used a static class called AssetManager that handled all the resources used by the game. This included textures, fonts, music, sound and levels.

The AssetManager class loaded an asset list on startup, which was a file that looked something like this:

# Asset File
TEXTURE ./res/tiles/tex_brick.png BRICK_001
TEXTURE ./res/crosshair.png CROSSHAIR

SOUND ./sfx/enm_hit.wav ENM_HIT

MAP ./maps/E1L1.grv E1L1

This uses a simple parser to read in first a type identifier, then a relative path, and finally an internal identifier that was used in-code to reference the assets.

These are stored in separate lists for each type of asset, and come with associated Getters (GetTexture, GetTexturePAK, GetTextureIndex, etc).

The system worked, but became really difficult to deal with every time I needed to add a new type to the manager.

I did a bit of thinking and decided to implement a simple VFS, or "Virtual File System".

What I wanted was a very simple abstraction that worked with a class called File, of which there were subtypes (TextureFile, SoundFile, and so on).

The VFS manager must maintain a list of all paths from its root, and allow for both a lazy-loading system and for a preload system, with hooks for loading screens if necessary.

With that, I started out with what I believed to be the closest 'skeleton' representation of the VFS class and its methods. VFS is a Singleton, but I'm ignoring the specifics below.

class GSVFS {
    public:
		void stageFile(std::string path);
		void preloadFiles(std::function<void ()> loaderCallback);
		
		GFile& getFile(std::string path); // Will load if not loaded, but its
										 // best to preload if possible.
	private:
		std::string root_path = ".";
		const std::string exclusions = "exe;dll;pak";
		
		std::map<std::string, GFile> files;
		std::vector<std::string> preload_list;

		void enumuratePaths(); // Recursively scan folders for files.	
							   
};

This is just a simple representation of what I have. The ability to stage

files for preloading is very useful when you're loading a lot of resources, even small ones.

My actual implementation has quite a few extra management methods (For clearing staged files, unloading loaded files, mostly).

It's quite a simple system, really. The only 'complicated' bit is the enumuratePaths() method. And that isn't really complicated at all. It uses the C library header dirent.h to operate on the directory.

Certain filetypes are excluded (Via the exclusions string).

In addition to handling files on the actual HDD, the VFS class can also manage PAK files. It treats PAK files in a slightly different manner, in that it adds one extra layer to the root path that matches the internal name of the PAK.

This has the interesting side-effect of allowing 'patches' to existing assets.

As an example, if the original release of Exile had utilized this system and I wanted to fix those holes in the maps without having to build and release the entire game again, I'd just release a "patch1.pak" that overwrites the faulty maps.

Additionally, the VFS class also handles general file IO. So writing new files or reading from text/binary files is managed at a relatively low level via this class.

The GFile class

Well, time to discuss the data the VFS actually deals with.

The GFile class is pretty much a wrapped ifstream with a virtual Get method that returns the file contents.

In the case of the plain GFile, it returns a reference to the opened ifstream.

Other variants I use so far are GTextFile (Returns a block of text), GTextureFile (Returns an sf::Texture) and GSoundFile (Returns a pointer to the beginning of a Sound for SFML).

The implementation for each is black-boxed so that the caller doesn't have to worry about how it gets the data, just that it'll get it (Or an exception if the file isn't the correct type).

The VFS class manages instantiating the correct types via a table of lambda functions mapped to extensions.

If an extension isn't covered, the default GFile handler is used.

Well, that's enough of that. I'm going to go find something to do. Work is slow today…

Comments