Thanks, I really receive good sentiments from the community. I've the feeling that this is not due just to the fact that many developers find Redis useful. It has very serious limits like many other softwares. It is that so far Redis was never proposed as "The Shit", the silver bullet that will save you from your data problem. Is just something I and the other contribs try to do with love, without pretending we are going to save the world. I wish this vision was a bit more common, but (un)fortunately data now is an hot topic with too much money on the table, so feelings are a bit over excited sometimes.
EDIT: Many marketing/business people basically don't understand that you cannot over-sell to developers. The effect is the contrary, you turn people down when you try to say you did something too cool to be true, so this strategy also does not work in the long run.
"without pretending we are going to save the world". I felt a great disturbance in SV, as if millions of voices suddenly cried out in terror and were suddenly silenced.
Just FYI, your comment got flagged but I vouched it. I think these little fragments of appreciation should be okay on HN. Sometimes an upvote just isn't enough.
This is it. Redis is one of the few things in the data-persistence space that is honest about what it can't do.
So many other DBs, caches, or whatever will promise that they can solve all my problems, making me quite angry when I spend weeks developing around them and find out that I've just wasted all that time because their marketing people lied to me.
Graph databases. They can handle your big data! Wait... did you say you had gigabytes of data? Multiple gigabytes? Oh dear, you're going to need to scale up to an enterprise cluster. What sort of madman would keep a whole gigabyte of graph data on one poor little computer?
Riak 1.x. Anyone remember Riak Search? Not the one in 2.x that offloads it to Solr (I don't know if that actually works), but the one before that, the one they deprecated before there was anything to replace it with.
Any database that promises that you can solve your problems with MapReduce. The developers do not actually want you to MapReduce inside their database. It will probably stall your database so hard it makes it unavailable. Their sales people made them put it on the feature checklist.
Antirez, I've accomplished a lot with Redis over the past six years, and I'm always looking for an reason to integrate it into whatever project I'm working on. Keep up the outstanding work.
Is just something I and the other contribs try to do with love, without pretending we are going to save the world. I wish this vision was a bit more common
So much open source is the result of, as is traditionally said, developers scratching their own itch. That's not to say that the developers don't start a project to "fix all the things wrong with the current options", but this is a more pragmatic view than the "this software is the final ultimate solution to problem X" that things eventually grow into having attached to them. I think a lot of proposing things as silver bullets grows out of the community as it gets more popular and is more the users proselytizing their flavor of the week rather than the core developers presenting it as something it's not.
The commercial side is, of course, a different beast, where salesmenship is required for any kind of longevity.
On the same note, I'd put nginx high up there. Aside from the occasional config funkiness (which I can forgive them for) it's an amazingly useful, versatile and efficient tool. Can't complain too much about it. I think nginx and redis are my two favorite often used pieces stack.
Redis has a great developer experience. It's super easy to spin up and to add to applications. There have been a few warts though when it comes to operational experience in production. I really miss things like administrative connections and tls for client connections that I get in the relational db world. Also, using redis in highly regulated environments is challenging because you have to meet all sorts of compliance requirements. As a result, I find myself jumping through a ton of hoops to get redis to work there. It'd be great if it "just worked" in that arena.
The answer to this question is one of the reasons I like Redis so much. It's not one thing, it's many things!
I've used it for a "dumb" cache (key => bytes), a "smart" cache (key => data structure), a message queue, a pub/sub engine, and even a lock manager.
For many of these use cases it's the best, for others, it's more than good enough and not having to add additional infrastructure components to a system is always a plus.
I like their new GEO sorted set with latitude and longitude. Can someone point me to a [reasonably priced] geocoding database to display nearest Town, Region (only where appropriate), Country (in English) based on latitude and longitude?
It's a big deal to run (takes lots of hardware), but I really like nominatim. I've played with them all and it's the one I've settled on. Helps that I run it on a pair of OSM tile rendering and routing servers. Only downside to nominatim is the space requirement (~ 700gb), and the fact that it takes forever to build the database (24 - 36 hours on 256gb ram machines w/ SSDs....)
I can second GeoNames. Sometimes it's hard to beat free. We take a couple files from their dump[1] (primarily the city data) and import into a database. We can then geolocate a lat/long to the closest city.
I'm using it for a photo app (I only need town/city level granularity), and it works well for my needs. Note that the dataset is based upon populated places though, so YMMV if you need to lookup somewhere in the middle of nowhere.
Here is a JavaScript library I wrote for it, along with some scripts for munging the GeoNames dataset:
I've been using RC for a couple of months now in production, albeit my site is very small, and has been proven rock-stable. And the code is a pleasure to read. Amazing piece of software! Keep up the good work!
Geo is insanely useful when you're after stuff within x miles of a known location - so a lot of mobile apps.
My problem with many existing geo implementations is they assume that a circular radius is the query type I care about, and longitude and latitude is sufficient to know where I actually am in terms of jurisdiction, etc.
Being able to add fine grained polygons to represent national, state, county or town boundaries and being able to identify the relationships between my co-ordinates and those entities is still far more complex than I would like.
Adding polygon support might be an interesting exercise, actually.
Hey Paul, we have this feature added as a Lua script, but adding it in a native way could be interesting actually. Itamar Haber wrote the Lua script and was pushing for adding it in a native way, I wanted to check after the release the actual demand for it.
This would be extremely helpful. We work with local governments and they have a cornucopia of geo-boundaries to query across. PostGIS works really well, but providing a cache-layer on top via Redis for simple counts would be amazing.
I work on a SaaS spatial product and had to come up with a solution to the same point/boundary problem. It needed to be 'fast' (ideally <3s) when filtering, rendering, styling and summarising 100k points within 43k polygons (so say 100k shop locations within many state, county and zip/zcta codes in US). DB is Oracle 11g for several reasons: product is larger and older than this requirement, small team has solely expertise with oracle and zero commercial experience with nosql dbs (good devs though and have showed interest in and toyed with nosql dbs) plus standard budget/time constraints.
Solution was to add a hidden column for each administrative category (e.g. for US: region, state, county, zcta) to each point record to replace spatial searches with filtering.
Would something like that work for you? What kind of numbers of points and polygons are you looking at and how quickly do you need to solve the problem?
> My problem with many existing geo implementations is they assume that a circular radius is the query type I care about
Really? Wouldn't most GIS implement something like OGC Simple Features? AFAIK all SQL GIS implementations follow that (to various levels of completeness, but even MySQL has shapes support since 5.6)
If you think to geo-aware mobile applications, many are of the kind "show me XYZ near me". So that's the main use case basically. According to the fact a lot of people switched to 3.2 while it was yet not stable in order to use this feature I believe there is a big demand.
Back 5 years ago, this would have helped us at our company. We were doing realtime data analytics and we had to dump our data to Postgres to run any geo queries.
The new BITFIELD seems like it could be really cool, but I can't quite figure out the use case. Anybody planning on using that that can enlighten me? Thanks.
I plan on using it to track what database objects have a corresponding cached image thumbnail. 10 million objects in my database, thumbnails generated on the fly, fast get/set the unique ID of the object in the bitfield. very small storage for what you get.
You can already do this using BITSET -- I tracked 100 million objects with incremental integer keys during a migration process and it worked great.
From what I read, BITFIELD behaves differently -- it treats your data as a series of "packed" bits. So while with BITSET you operate on each bit individually, with BITFIELD you operate on a series of bits with a starting offset, and you have to be mindful of the length.
To expand on that topic: You can find all the Dockerfiles of the official images linked at the very top on Docker Hub [1]. Once my pull request is merged a Docker-Redis maintainer will have to open another pull request to update the references of the official image on Docker hub [2].
In fact I already updated the Redis Docker image once. You can see the full process there [3].
Kudos to antirez on everything before this and another great release.