I'm starting from scratch

[LucaMs]

"I'm starting from scratch" is a non-literal translation of the Italian phrase: "Ricomincio da zero" ("I restart from zero", literally).

A famous Italian film was called: "Ricomincio da tre"; the protagonist, when someone asked him why "da tre" (three) and not "da zero", said that three things in his life had worked out well for him.

After this long introduction, I decided to "get seriously" in the development of a complex project that I conceived 8-10 years ago but in a very indecisive and inconsistent way (a few days every year!), hoping that this time I won't immediately feel like giving up.

I begin by submitting a question to you (next post).

 

[LucaMs]

The project is client-server; I have to develop the server.

First question/doubt:
The current state of each connected user and other objects related to him.

I had thought (and implemented in many versions of the server) NOT to save them on mass storage, especially for a question of overall speed. Access to hard disks is slow. I am an old programmer, so I do not always have SSD mass storage devices in my mind and in any case these are certainly much slower compared to access to the central memory, commonly called RAM .

It is obvious that in the event of a server crash, perhaps due to a power failure, the data not saved on mass storage would be lost. Is it worth the risk and avoid weighing down the server by making it continuously access hard disks / SSD?

 

[epiCode]

Following three things will define:
1. 'In the event of a server crash". how often is that event?
2. How critical is the data loss?
3. How critical is the response time/speed or what is actual slowdown if regular read write operations are done?

Also you can use cloud storage which are scalable and have fast response time even in heavy load scenarios.

 

[LucaMs]

Following three things will define:
1. 'In the event of a server crash". how often is that event?
2. How critical is the data loss?
3. How critical is the response time/speed or what is actual slowdown if regular read write operations are done?

Also you can use cloud storage which are scalable and have fast response time even in heavy load scenarios.

1. Who knows?
2 Who knows?
3 Who knows?

(4) Cloud = other servers that will also have to write to mass storage devices.


Point 2: I am actually the one who should evaluate how important that data is.
These are games, so users could lose virtual money or scores in ongoing games.

 

[Sandman]

Premature optimization is the root of all evil (or at least most of it) in programming.

- Donald Knuth (info)

 

[Jeffrey Cameron]

We wrote a cloud-based point-of-sale application utilizing Android tablets for the "register" portion. The application makes requests to our ASP.NET back-end via a custom API we implemented. The tablet holds non-volatile semi-static information (e.g. employee list, time-clock list, etc.) but everything else is "real-time" requested from the server which queries MSSQL server databases.

Given that context, our metrics indicate each request averages less than 200ms response time, with tens-of-thousands requests from all users in any given month.

My advice? Don't worry about storage access times or storage space. Those can be easily scaled if necessary. Focus more on economy of data transmission between client and server.

 

[LucaMs]

queries MSSQL server databases.

I realize that I have asked the question incorrectly, which could be interpreted as if I wanted to decide whether to save objects on mass storage (using serialization).

Obviously I am referring to any saving of "objects" on DBMS and consequently on mass storage.

"To save or not to save" all on mass storage (DBMS).

 

[hatzisn]

1. Who knows?
2 Who knows?
3 Who knows?

(4) Cloud = other servers that will also have to write to mass storage devices.


Point 2: I am actually the one who should evaluate how important that data is.
These are games, so users could lose virtual money or scores in ongoing games.

Follow Elon Musk's example for starship. Crash a lot - fly once. Just kidding. Redundancy is the answer. I was reading somewhere that the way starship flies is that it uses three x86 dual core (???????????) processors that run redundantly in a special computer that runs a linux distro created by Space X (both). These processors, all 3 redundantly make the same calculations and compare their results continuesly. They all act redundantly and in case of one's failure the next takes on. I though it was a joke and maybe it is just a gossip, but after all he is Elon and he does not think the same way as most of people. So this is the answer. Have at least 2 vps constantly exchanging mqtt messages with json and you may make your game fly. If one is down the other will be up. Save every X minutes.

 

[LucaMs]

Have at least 2 vps constantly excanging mqtt messages with json and you may make your game fly. If one is down the other will be up

 

[hatzisn]

Premature optimization is the root of all evil (or at least most of it) in programming.

- Donald Knuth (info)

Was he drunk? Plan, create, check, correct and around we go. If correct is a bottleneck the cycle becomes an arc.

 

[Sagenut]

Was he drunk? Plan, create, check, correct and around we go. If correct is a bottleneck the cycle becomes an arc.

I think that what @Sandman is trying to say is:
If you pretend to prevent all the possible problems and scenarios that some code can create you will end up like the dog spinning around trying to bite his own tail.
Of course every good developer will try to think to the majority of problems and prevent them, but nailing the perfection at the first shot it's nearly impossible.
At a certain point you should publish your creation and then wait for the users feedback.
On that you will improve, add, correct.
Seeking the perfection (only to your eyes at that moment) could lead you to an eternal do/undo/redo cycle.

 

[LucaMs]

but nailing the perfection at the first shot it's nearly impossible

Yes, but in this case it's not about going into details, it's not about precision but about basic, fundamental choices.

 

[hatzisn]

I think that what @Sandman is trying to say is:
If you pretend to prevent all the possible problems and scenarios that some code can create you will end up like the dog spinning around trying to bite his own tail.
Of course every good developer will try to think to the majority of problems and prevent them, but nailing the perfection at the first shot it's nearly impossible.
At a certain point you should publish your creation and then wait for the users feedback.
On that you will improve, add, correct.
Seeking the perfection (only to your eyes at that moment) could lead you to an eternal do/undo/redo cycle.

Maybe the language I used is kind of offending now that I read it again. So @Sandman I apologize. Experience tought me that correcting afterwards done mistakes takes much more time and thus money than planning ahead. But I believe that maybe the final solution must be hybrid between what @Sandman suggests and my aproach. And this is the agile development.

 

[LucaMs]

Maybe the language I used is kind of offending now that I read it again. So @Sandman I apologize

No, it wasn't offensive and certainly not towards @Sandman, but rather towards that Knuth and his habit of drinking alcohol

 

[Jeffrey Cameron]

I realize that I have asked the question incorrectly, which could be interpreted as if I wanted to decide whether to save objects on mass storage (using serialization).

Obviously I am referring to any saving of "objects" on DBMS and consequently on mass storage.

"To save or not to save" all on mass storage (DBMS).

You were clear, I could have done a better job of explaining....

When the "register" tablet talks to the API, everything that happens is immediately written to the databases, the tablet receives a complete new image of the ticket it is creating after every request. So, even if the tablet or the server crashes, it will pick up right where it left off when it comes back up.

Even with all of the writes to databases, and adjusting of the ticket and its items, each API request takes less than 200ms on average.

 

[LucaMs]

You were clear, I could have done a better job of explaining....

When the "register" tablet talks to the API,

Well I don't know what you mean by "register" tablet, but I'll detail the requirements a bit more (repeating, perhaps, that it's a very old project, of which I've already developed many versions, none of them satisfied me and all of them kept most of the information only in RAM - I'm talking exclusively about the server).


I have a map of rooms (and here I have various problems, including the lack of inheritance), a map of connected users, user-room-game associations, etc.

At any time a user can perform an action (even non-game), so he will change his player status and that of the game and the room.

All this was like this in all the old server versions, almost without anything written in the DB.

It is obvious that in case of server problems (crashes, power failure and other) it would result in the loss of data from the games in progress.
I fear that saving every action, every state of every "object" on DBMS and therefore on mass storage (hard disk or SSD) the server would slow down too much (hoping to have 2,000/3,000 users connected at the same time).

Is it better to risk and store everything in central memory or write everything to the DB?

 

[Jeffrey Cameron]

By "register tablet" I mean the device the clerk is using to ring up a customer's sale, in our case a 10" Android tablet.

I guess it depends on how much data you need to save. if it's a few bytes per user then it shouldn't be a problem saving it as you go. If it's a large amount of data, I'd probably write a separate "save data" thread that would save "snapshots" of what's in RAM periodically, how frequently determined by balancing potential data-loss against server load.

Alternately, you could have a managed "queue" of database updates that the server updates in addition to it's RAM table when a user performs an action, and a separate thread that processes that queue. This would de-couple the database updates from the actual server handling.

 

[OliverA]

I would save to disk/use the disk. Even "slow" disks are often not the bottleneck, it's usually the transmission medium between the client and the server.

 

[LucaMs]

So many posts! (Thanks to all)

And this for one issue only; the other 5,973 are still missing