Historical Archive

Last month me and Sean have talked to several of our users over video calls. One of our goals was learning about frictions you might be experiencing, and ways in which to mitigate them.

One complaint was that you’re not getting any results, or that you’re getting too many irrelevant results. We’ve addressed that with two guides, though admittedly, the approaches outlined in them require significant time and effort. Can we do anything to predict how good a filter will be immediately?


Well, Not Immediately

You might be wondering how come we’re the only ones out there with filters capable of e.g. matching a question mark in a post title. We achieved that by foregoing search engine functionality and only focusing on live content matching, meaning we don’t need to store an index. It’s extremely disk-consuming to create an index for something as seemingly simple as prefix matching, not to mention exact substring matching. We’re talking hundereds of times of more space of the original item, quite unrealistic. And doing matching on an archive without an index is slow.

But a slow archive is better than no archive. We’re storing one month of historical content, and when you define a new filter we’re matching each and every item one by one. Just reading that much data from disk takes time! You can see the preview of our search in the Setup page at any time.

This process can take several hours. Worry not though, we will send you an email as soon as the search is complete.


Conclusion

We’re constantly improving our product, so stay tuned and don’t hesitate to tell us about your frictions. Until next time!

See who mentioned you in the last month Create an Account