Half a year ago I implemented newsletter signups on this website, using Mailchimp. They gave me a code snippet to copy paste onto this website, which had some honeypots (fields in the form that humans should not fill) for protection against bots.
Then I activated one setting on Mailchimp. It made an email with captcha to be sent to the user before actually adding them to the list. It worked and prevented the bot signups to the newsletter. Good ending right? well... no. After some weeks I began to notice that my domain reputation went significantly down, and the mails I send from my domain mailbox consistently appeared in receiver's spam folder.
That's when I started to think about malicious bots. Those that scrape my content, steal email addresses or just trigger signups. The Mailchimp protection was not enough, I had to find another solution.
There are various Captcha solutions available. From Google, Cloudflare and others. I tried to implement the free Cloudflare captcha and it was kind of broken at the moment.
I realized that there are many cons in captchas:
Second, there are rules of when you have to show on your website a cookie consent window, which is something I really try to avoid as it harms experience. Captchas often require this consent window to be shown.
I decided to avoid the Captcha solution.
I came to an understanding that websites contain assets that you have to protect. Your content, the email address or the form signup (wasn't obvious for me at first). Cloudflare has some amazing tools for bot protection, email obfuscation and others, so I enabled some of them.
First, we need to understand what bots we are facing. For an unpopular site such as mine, the malicious bots that I meet are those that just scrape content. No one will spend resources on smart, content loading or AI bots on my domain. Another enemy are bots that fill forms and click on submit buttons (the one's that created the issue in the first place and made the domain value to go down)
Second, what are the assets? My mail address to not get spammed. The website content so bots won't copy paste it into lower quality places. My domain and mail reputation that can be harmed on fake newsletter signups.
Now, when we understand the enemy and what we protect, let's think what to do to defend ourselves from possible issues:
Email address. If my mail address will be in plain html the bots will be able to fetch it. Also, I guess it's not that hard to find strings where the @ sign is replaced, like MY_EMAIL (at) gmail.com. The solution is to have the address in the website content hidden (encrypted). The decryption is made using javascript code (that the bots usually won't execute on an unpopular site such as mine).
Form sign up (newsletter in my case). While the Mailchimp honeypot blocks the less intelligent bots, some bypass it. I added other custom heuristics in javascript that try to detect a bot and only after passing all of them the sign up url is decrypted and the form submit option becomes available.
A good practice is to be able to answer on the question: What if?
What if some bots bypass this sign up protection and trigger my mails? while I don't expect it to happen often, I reduced the risk of reputation damage by separating subdomains and mailboxes for various email types.
What if a bot is able to fetch my main mail address from the website despite my protections? I will get some spam, most of which will be blocked anyway.
Let's go over the logic. View the javascript code in the attached images below.
As you see in the images, the tests aren't that complex: If honeypot filled it's a bot. If the visitor failed to type 'I am human' 5 times it's a bot. If the visitor filled and submitted the form too fast it's a bot.
If bot is detected the relevant flag is set and doing page refresh required to try to subscribe again. Only after passing all the tests successfully the required data is appended back to the html to request signup. Feel free to play with it on my newsletter form!
We talked about, sometimes not so obvious, but important things. Website assets and reputation protection, with methods to reduce risk and protect yourself with no captcha.
6 months passed since the event. There are zero bot signups to the newsletter, my domain and email reputation went back up, and seems like no bot defeated my custom defense yet.
The main takeaway is that you should protect your website and email value. Hopefuly the experience, tips and methods I share with you in this guide will help.
Feel free to contact me or ask questions. If you gained some insights this guide was not in vain :)