I am a one man army right now in my small indie game studio (GamerzDan) out of India (Registered in US as GamerzDan, Inc.), so I generally do not put my time to create these kind of guides or tricks/tips as a programmer or YouTube videos, I use most of my time prototyping and programming in general.
But recently due to my need of a backend database for our upcoming Chess Game "Shatranj" and my interest in going for a Cloud solution instead of a self-hosted one (server+mysql+php/api) which I normally self-create and use, I decided to look into Google Firebase, which is a suite of Backend as a Service products for apps and in general.
Now, I have known and been aware of Firebase since years even before being into game development, or app development before it. But I never actually ended up using it for any of my client or production projects so this was my first time actually diving into it.
Table of Contents
- Introduction to Realtime Database and Cloud Firestore
- Differences between Realtime and Firestore Databases
- Pricing model of Realtime and Firestore Databases
- Code Difference and Approach between Realtime and Firestore Database
- Followup Parts of this guide
Introduction to Realtime Database and Cloud Firestore
I was interested in Firebase's cloud database solutions, of which the most popular being Realtime Database and other more powerful, newer and state of the art solution being Cloud Firestore.
Due to lack of general information available for Cloud Firestore and Firebase's docs seeming quite intimidating, I looked into Realtime Database first for my specific needs but pretty soon realized its not for me due to how it works, how the data is structured (JSON eww) and some of its other limitations I felt from my understanding, compared to which Cloud Firestore just seemed a perfect replacement for anyone currently using or being used to old school RDBMS like MySQL.
Difference between Realtime and Firestore Databases
I have discussed the major differences in my attached Youtube Video (again, as per my understanding and needs) and why as a GameDev or AppDev in general you should avoid Realtime Database and go for Cloud Firestore.
But as a refresher, differences between Realtime Database and Firestore and why to go for Cloud Firestore as a game or app developer:
- Realtime Database uses a JSON tree-like nested structure, your complex or large data can get really branched out or nested, harder to read for yourself and becomes hell remembering the structure while accessing it. Overall, its way different than your standard SQL database structure.
On other hand, Cloud Firestore uses a more SQL-relatable data structure which is key-value based (just like mysql columns and rows) and uses dictionary as the standard way to send or get data to your app. Furthermore, Firestore divides your whole data structure into Collections (Tables), Documents (columns/data header) and then fields (rows/actual data). Its easier to understand, analyze and plan your database structure and thus, saves you valuable development time as you feel more used to it and easy to remember. - Code difference, this depends on personal choice but I like key-value dictionary structure more than JSON structure, and thus, like how Firestore's code works to send or get data to your database. It feels more natural to write the code and prepare your data when its in a dictionary.
- Search Query/Data Filtering, again depends on personal choice but Realtime Database plain and simple sucks for filtering data or querying on it. You can only run one Query per call and cannot club multiple conditions as easily as you can in Firestore with a single call, furthermore, Firebase themselves pride Firestore to be able to run complex and compound queries on millions of data within a second.
- Data Types. Realtime Database is a flat json text data, cannot support Data Type, boo. Firestore supports almost all standard Data Types and even your custom classes, yay.
Pricing model of Realtime and Firestore Databases
Both Realtime Database and Firestore has free usage tier, which is pretty generous and chances for most apps/games are you don't need to pay a penny till your game grows and goes big.
Firebase uses a Pay-As-You-Go pricing model which is neat as even for paid limits, you do not need to pay fixed costs per month and only pay as per your usage and needs (can be a few cents or a few thousand dollars).
Realtime Database has a pretty straightforward pricing model.
- You have a simultaneous connections limits i.e. how many apps/users can connect and run queries at once. It is 100 in free tier and 200k/database in pay-as-you-go tier.
By the way, that's 200k connections PER DATABASE and not per account or app, so if you are smart and pre-plan your database structure, you can spread your one big fat database into multiple databases to hold common data (one db for users, one db for matches, one db for logs, etc.) and thus can achieve even 1 Million connections at once (in real world your gonna need when you make a hit like Dota 2 or PUBG). However, in free tier you can only create 1 Database, unlimited in paid. - Mostly you only pay for amount of storage your data needs (1GB free tier and $5/GB paid) and amount of outgoing traffic (10GB/month free, $1/GB paid) your app uses (Firebase to your app, or data you retrieve). There is no cost for incoming data towards Firebase (Your app to Firebase, or storing data). That's sweet.
That data storage of 1GB is quite sufficient for most devs even across multiple projects. My bloated wordpress site's database uses 600MB~ and it has 150k registered users and more than 3 Million rows of data. - Only thing of actual interest is that outgoing traffic, which again, is quite sufficient even in free tier but can get out of hand if you plan your database structure or program your code poorly (like getting data every frame or in update loop). That traffic cost can cost you a fortune if your really using high amount of data, just 10GB of network can cost you $10 and as a example, that same bloated wordpress database I mentioned above uses 250GB PER DAY (so in Realtime Database terms, thats $250 per day). Now obviously wordpress is a bloated platform, database is not as optimized and its a large database but you get the picture. As I self host it myself (server+mysql), I really only pay around $10~ PER MONTH. I mean, if I were hosting that same database on Realtime, I would be using 7720GB network per month, so $7k charge.
Now comes the more interesting one, Cloud Firestore pricing model, not as easy to understand at first.
- There are no connection limits in Firestore, 1 user, 10 users or 10 million users, Firestore can handle it all.
- Just like Realtime, we get 1GB storage in free plan however, we only pay $0.18/GB for extra storage we use. Comparing it to Realtime's $5/GB, this is dirt cheap.
Now there is a argument as Realtime uses flat text data in json and Firestore uses Data Types and key-value, Firestore uses more storage for same amount of data compared to Realtime. That is true. But lets be honest, how much would extra storage would that be ? 2x times? 5x times? Even with 5x times, you pay $0.9 for 5GB of storage. Some articles make entry level users really paranoid with Firestore costs, but when you calculate it using Real World use case and examples, one can even argue it costs less than Realtime. - Just like Realtime, we also pay for network, only outgoing traffic (data retrieved) and not incoming traffic towards firebase (data store/sent, its free).
We get 10GB/month traffic in free tier, but paid traffic has a bit more complex structure as it uses Google Cloud pricing for traffic which depends on the location your Firestore database is hosted (US, EU, Asia, etc.)
However, as a real world example, lets suppose we store our Firebase Database in Europe (EU) region as I consider it more central to handle Asian and American user traffic.
* All incoming traffic towards Firebase servers is free.
* All outgoing traffic from Firebase towards your users/apps in same region is Free i.e. all traffic to all our users in Europe is free
* All outgoing traffic from Firebase towards users in US is charged flat at $0.01/GB and this is regardless of which primary region you host your database, so if you chose Asia you pay same rate for all traffic used by US users. However, if you choose US as your database location, you pay nothing due to same region rule.
* All outgoing traffic from Firebase towards users in any region except US or your database region (EU) costs you between $0.1-$0.2/GB depending on which country/region the user is from. For example, Singapore is $0.1/GB while China is $0.2/GB
Even if you consider the higher limit of $0.2/GB, its still cheaper than Realtime cost of $1/GB. Basically in same $1 amount, you get 5GB network (minimum) to 10GB or more depending on your user's location and which location you decide to store your Firestore database in. - Other than network, you also get free usage and paid usage for amount of Read, Write and Delete operations you make on your database. These limits are per day and fairly generous.
Read operations are 50k/day free and $0.06/100k paid
Write operations are 20k/day free and $0.18/100k paid
Delete operations are 20k/day free and $0.02/100k paid
Taking my chess game as a example, if I have 500 users online and playing at per day (250 matches) and each match I need to send/store 100 moves on average, I need to make 250 x 100 = 25,000 write operations each day, which are almost within my free tier limit. Even when I cross the 20k limit, I am paying $0.06 for the next 100k I use. Do note that the above example uses extreme write operations because I know for a fact I won't be making 100 move calls per match and if I need to, I can just store all the moves in a large dictionary and make a single call to send all that data at once after the match ends instead of 100 individual write operations.
Believe me, free tier is very generous and chances are you will hardly need more than that. Furthermore, here is icing on cake, any and all pay as you go charges are ABOVE and AFTER your free tier limit.
If you use 20GB bandwidth a month, 10GB will be free as part of free tier and you pay for 10GB.
If you make 25k or even 50k write operations a day, 20k of those will be free and only the difference will be counted from the 100k you get for $0.06 (which btw is not even per day and as a total amount of credits whether you use 10k from 100k per day or per week).
More Google Cloud Info on Pricing here -
https://firebase.google.com/pricing
https://cloud.google.com/firestore/pricing
https://cloud.google.com/firestore/pricing#internet-egress
Code Difference and Approach between Realtime and Firestore Database
The reason many entry-level devs avoid Firestore and why there is such a huge lack of tutorials, guides, examples or resources on using Firestore with Unity, is how confusing Firebase's docs for Firestore are and how bad their actual code examples are. They are even bad for a boilerplate code and they should have provide some better examples and explanations for their code examples, heck even a example use case.
However, once you start getting your hands dipped into Firestore and understand how it works, how you interact with your data and its structure, how to structure your data internally in your code, Firebase is actually way way easier than Realtime Database and even SQL databases in general, once you understand and start restructuring Firebase's code examples to your own needs, it becomes a breeze to use and now I don't think I will even ever use MySQL again unless I specifically need to (example: my firebase costs start getting out of hand and I know that my costs for a self-hosted MySQL database and server will be lower).
Boilerplate Code Example of using Realtime Database in Unity3D (C#)
Add/Save Data to Realtime Database
mDatabaseRef.Child("users").Child(userId).SetRawJsonValueAsync(json);
mDatabaseRef.Child("users").Child(userId).Child("username").SetValueAsync(name);
Get/Retrieve Data from Realtime Database
FirebaseDatabase.DefaultInstance
.GetReference("Leaders")
.GetValueAsync().ContinueWith(task => {
if (task.IsFaulted) {
// Handle the error...
}
else if (task.IsCompleted) {
DataSnapshot snapshot = task.Result;
// Do something with snapshot...
}
});
Filtering Data (Query/Search) in Realtime Database
FirebaseDatabase.DefaultInstance
.GetReference("Leaders").EqualTo("score").LimitToLast(1)
}
^^The EqualTo that checks for score on what or where, no explanation provided, heck even above code example is created by me as Realtime Database has 0 actual code snippets on Filtering Data:
https://firebase.google.com/docs/database/unity/retrieve-data#filtering_data
Boilerplate Code Examples of using Cloud Firestore in Unity3D (C#)
Add/Save Data to Cloud Firestore
DocumentReference docRef = db.Collection("cities").Document("LA");
Dictionary<string, object> city = new Dictionary<string, object>
{
{ "Name", "Los Angeles" },
{ "State", "CA" },
{ "Country", "USA" }
};
docRef.SetAsync(city).ContinueWithOnMainThread(task => {
Debug.Log("Added data to the LA document in the cities collection.");
});
Get/Retrieve Data from Cloud Firestore
DocumentReference docRef = db.Collection("cities").Document("SF");
docRef.GetSnapshotAsync().ContinueWithOnMainThread(task =>
{
DocumentSnapshot snapshot = task.Result;
if (snapshot.Exists) {
Debug.Log(String.Format("Document data for {0} document:", snapshot.Id));
Dictionary<string, object> city = snapshot.ToDictionary();
foreach (KeyValuePair<string, object> pair in city) {
Debug.Log(String.Format("{0}: {1}", pair.Key, pair.Value));
}
} else {
Debug.Log(String.Format("Document {0} does not exist!", snapshot.Id));
}
});
Query/Filter/Search Data in Cloud Firestore Database
CollectionReference citiesRef = db.Collection("cities");
Query query = citiesRef.WhereEqualTo("State", "CA");
query.GetSnapshotAsync().ContinueWithOnMainThread((querySnapshotTask) =>
{
foreach (DocumentSnapshot documentSnapshot in querySnapshotTask.Result.Documents)
{
Debug.Log(String.Format("Document {0} returned by query State=CA", documentSnapshot.Id));
}
});
Point to note is that even the above provide Code Snippets in Firebase Firestore docs are quite unexplaining and can be confusing to entry level devs switching from MySQL or other databases.
However, once you start understanding how the data is structured in Firestore, how to structure your own data, which Firestore methods to use for what purpose, you will realize that actual real world use code of Firestore is quite simple and human readable.
We will see more actual code examples that I have created and using in our own Chess Game Shatranj in followup posts but here is a example:
My way of adding new data to my Moves Collection/Table, I am passing the document name I want to use as matchid and a Dictionary<string, object> userData with actual data to send and store.
I have saved references to both FirebaseFirestore object and my moves CollectionReference in moves object so I can use a single keyboard to access that instead of db.Collection("moves").Document(matchid)
internal void fsAddToMoves(string matchid, Dictionary<string, object> userData)
{
DocumentReference docRef = moves.Document(matchid); //This should always be unique or else will be overwritten
//Show loading bar UI
docRef.SetAsync(userData).ContinueWithOnMainThread(task =>
{
//Executed when updation done, probably disable your loading progress here
//Disable Loading bar UI
if (task.IsFaulted)
{
Debug.Log("Updating move failed " + task.Exception.InnerException.Message); //probably userdata.userid is blank
}
else
{
Debug.Log("Moves updated: " + docRef.Id);
}
});
}
Boilerplate Code version
void addData(documentTitle, data){
moves.Document(documentTitle).SetAsync(data).ContinueWithOnMainThread(task =>
{
//Task completed, check if it finished and data saved success
}
}
Followup Parts of this guide
Part 2/4: Adding/Saving Data to Cloud Firestore
Part 3/4: Getting/Retrieving Data from Cloud Firestore
Part 4/4: Quering, Filtering and Searching Data and Nested Data Structures
