Phew, It’s nearly time I implemented a Game Database!
I mean, using JSON files for holding data is great but working with it
in the format provided isn’t.
Loading each file and converting it to JSON and then loading another one
and then trying to combine them all makes it extremely difficult to code
something worthwhile.
You might be thinking: Yeah, I’m going to use (no)SQL for the task! Nope, SQL is not up untill later in the development process.
I first need to import all 606 JSON data files. Convert eachone of them
to a Elixir data structure. Then iterate over each data structure list
and all it’s objects.
Then add each one of them to a unified Map/Structure in memory database
which can be searched.
After the script is done done it can then save this database to a file. That file will be loaded into a process and accessed afterwards throughout the development process..
Later I might convert it to a SQL database or something similair so we
gain development speed.
But frankly, the files are 10 MB. If we use around 10-20MB worth of RAM
then it’s really no need to complicate things with a database for the
JSOn files.
Removed any non json file from the data/json folder (including some
extracts from thes source code)
Moved (and modified) data/json/uncraft/classes/ammo.json as it
misbihaved and crashed our server.
The current object handling implementation is pretty simple
Current object handling implementation
def handle_object(object) doid = object["id"]type = object["type"]abstract = object["abstract"]IO.inspect objectif !is_nil(id) or !is_nil(abstract) do\tif is_nil(id), do: id = abstract\tcreate_category_for_type(type, %{})\tDataStorage.add(@database,[type,id], object)else create_category_for_type(type, [])# IO.inspect object DataStorage.append(@database,type,object)end end
WOW. I didn’t believe it. But almost 10 MB of json was translated to 1
file of 347.5 KB with 1 big elixir map.
I still have to look into it to view if everything is there and nothing
is missing as I suspect something might have gone wrong:)
Yup, upon looking into it.. it seems that data is TRUNCATED when we try
to save it as a map.
Seems that saving it with :erlang.term_to_binary works and the file size
is now 7.9MB
It could have been larger but term_to_binary actually zips the data ☺
Upon further investigatin I found out that I can use a special setting
to allow infinity data to be printed out.. Great.
I guess I’ll save both term and map data for the time being.
IO.inspect(large_string, printable_limit: :infinity, limit: :infinity)
And it’s now 8.7 MB .
By looking at the JSOn source I can see that the uncraft type is
special and that it contains a result key which could take as sub ID
for the uncraft key instead of using it as a list.
I’ve also noticed that it’s interesting to select the ID and be able to
show the user a name instead of the id. This will be useful when
searching for names to show the user for nondescript ID’s.
Let’s just store the id and name reference in a different key I’ll call "names"
For now I’m content with the current Database I’ve imported and saved to
file. It could be better but I can now start prototyping other aspectsof
the game.
You can view certain changes here
https://lba.im/fossilr3po/lifebeyondapocalypse-elixir/info/135070ed01d6e6ec
Next we’ll look at an improved and extended item searching functionality