As someone who helps keep a site online with a lot of content, I have mixed feelings on Firecrawl.

On one hand, their bots seem much more well behaved than others.

However, running a crawler fleet which is deceptive and evasive in its identification and don't honor REP is no way to build a business.

I'd love for you to kick the tires on https://grubcrawler.dev

"they copied my idea by showing interest in my product and then copied it". What exactly is revolutionary about Firecrawl or your product? Scraping APIs have been around for over a decade.

I was the first to return markdown and use reader mode stuff to strip irrelevant stuff. Theres copying and there's talking to the founder sounding interested to have your team copy what I did in the background. One is fair game, the other is a dick head move.

Not sure about the first claim. But yes, talking to the founder, sharing details and having it stolen is not a good look. Sorry that happened to you.

I think that is a neat idea and it sucks this happened, but how long before somebody simply saw that feature and replicated it? I'm curious, had you considered a deeper moat than that?

This is especially relevant given AI is making this kind of thing easy at an industrial scale. I think we should all be looking for alternative moats.

Sometimes timing is your moat and that's all you need. That being said I'll probably start limiting my public releases to revolve around standards I want implemented.

I'm rethinking the sources of value moats are built around. It seems like the landscape is changing and dimensions such as location, perspective, experience, and attention weigh more than they used to.

> but how long before somebody simply saw that feature and replicated it?

This is a good example. The, idk, "value store" of your org just switched from products and services to the employees who understand your process from a couple angles and can write well.

Tell more. Crawling is not a new idea. How did they abuse you?