Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

The number of bits you need in a bloom filter varies as a function of the number of items you have (in addition to the desired error rate). Assuming a 100 items, a 1e-8 error rate only requires 480 bytes (= ~5 bytes per item).

You also don't need to do a lot of independent hashes. You can just take a linear combination of the results of two hashes and get basically no performance degradation (see: http://www.eecs.harvard.edu/~kirsch/pubs/bbbf/esa06.pdf).

Also, you probably don't actually need a 1e-8 false positive rate, since you're going to fall back to a canonical (but slow) lookup table for positives in most use cases. Bloom filters are most useful when your real dictionary is slow and you're expecting a lot of 'no match' results.



> Also, you probably don't actually need a 1e-8 false positive rate

Absolutely not. I just wanted to demonstrate that even if you do want a very low false-positive rate from your filter you don't need a very large data structure, contrary to jeresig's fears. Bloom filters are very, very space-efficient.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: