Bringing up Redis, Riak, and ElasticSearch is nice...
You see, ElasticSearch gets split brain surprisingly easily, if you have a real significant load / number of hosts / host failures you will most definitely experience it, whereas with most other databases it'll only happen once every couple of years to the very largest and heaviest users. It's also just not known for data consistency, you just have to be able to refresh data from some other canonical source periodically.
Redis requires 2x memory present as is used to serve the load of requests, in order to backup. You can only safely really use half the server's memory. Because it has to fork to either save the snapshot or to compact the write log.
Riak performance is difficult to manage and keep consistent, as you replace failed nodes.
What I have experience with is a very large / heavy use of mysql at a popular website. We did manual/application sharding, and master/slave streaming replication, with automatic read-failover to slave, but only manual promotion. The slave did backups each day, and there was enough transaction log on the master and slave to spin up a new slave to either, with a backup up to 3 days old, and have it catch up. The nature of the load was that it was not possible to take a backup of the master in order to seed a new slave, because the traffic never stops and we could not afford the memory or iops overhead of backing up the master, and we really didn't do downtime if we could help it. A couple hours a year maybe.
We had trending and alerting of everything, including replication, roles, and we automated it all with something not unlike the configuration it appears you have to use for postgresql. Frankly these new-fangled auto-everything databases are untrustworthy, I've seen three of them fail (in ways you might not notice if you're not a graybeard), and the fourth has a name that really puts me off. RethinkDB? yeah, go ahead kids. I just love all this unreliable crap on the internet these days...
Frankly these new-fangled auto-everything databases are untrustworthy
I'm not asking for postgres to change its ways and become "untrustworthy".
I'm merely asking that postgres wraps more automation and convenience around the things that it already does.
Replication should generally be operated from inside psql. Spinning up a slave should be a single command. Failing over should be a single command. There should be meaningful error messages, ETAs and progress bars. No rsync'ing of WAL files, no futzing with config files. No elaborate third party 'repmgr' daemons.
> I'm merely asking that postgres wraps more automation and convenience around the things that it already does.
and if history is any indication, it will when it's good and ready to be convenient and automated. as a previous poster (or two) most eloquently noted, postgres tends to work from the bottom up. when the "fundamentals" have been hashed out and thoroughly tested through all or nearly all edge cases, then convenient interfaces "bubble up"
You see, ElasticSearch gets split brain surprisingly easily, if you have a real significant load / number of hosts / host failures you will most definitely experience it, whereas with most other databases it'll only happen once every couple of years to the very largest and heaviest users. It's also just not known for data consistency, you just have to be able to refresh data from some other canonical source periodically.
Redis requires 2x memory present as is used to serve the load of requests, in order to backup. You can only safely really use half the server's memory. Because it has to fork to either save the snapshot or to compact the write log.
Riak performance is difficult to manage and keep consistent, as you replace failed nodes.
What I have experience with is a very large / heavy use of mysql at a popular website. We did manual/application sharding, and master/slave streaming replication, with automatic read-failover to slave, but only manual promotion. The slave did backups each day, and there was enough transaction log on the master and slave to spin up a new slave to either, with a backup up to 3 days old, and have it catch up. The nature of the load was that it was not possible to take a backup of the master in order to seed a new slave, because the traffic never stops and we could not afford the memory or iops overhead of backing up the master, and we really didn't do downtime if we could help it. A couple hours a year maybe.
We had trending and alerting of everything, including replication, roles, and we automated it all with something not unlike the configuration it appears you have to use for postgresql. Frankly these new-fangled auto-everything databases are untrustworthy, I've seen three of them fail (in ways you might not notice if you're not a graybeard), and the fourth has a name that really puts me off. RethinkDB? yeah, go ahead kids. I just love all this unreliable crap on the internet these days...
PS I'm 26 :)