TL;DR: Have you ever thought about how to protect your application from spammers? Have you ever tried to prevent undesirable content injection? This tutorial will introduce you with Auth0 IP Signals, a free service from Auth0, and will show you how to use it in a spam detection use-case. The full code you will write throughout the tutorial is available at this GitHub repository.

The Spam Problem

In the beginning, spam affected emails. Receiving hundreds of unsolicited messages caused - and still causes - many inconveniences for email users: from the time spent to remove unwanted messages to the risk of phishing and similar traps. Today, the spam problem does not concern just email clients and servers. It also affects other media: from instant messages to Web search engines, from blogs to wikis, from social networks to e-commerce websites.

Regardless of the media involved, spam costs a lot both to companies and to individuals. It wastes your time and productivity, consumes system resources and drives up the cost of IT operations, can hurt the company's image, and jeopardizes security.

Why do they spam?

You may wonder why spam exists. Why do people send messages and content flooding our mailboxes, social media, or blogs? These are some of the most common reasons:

  • Promoting a product. This is probably the main reason for the original mail spamming. It aims to advertise a product, a service, or something else by reaching as many people as possible with the least expensive email marketing campaign.
  • Increasing SEO. This is the typical goal of the content injection in blog comments, wikis, forums, and so on. This activity is known as content spamming. It tends to abusively create HTML content with backlinks to e-commerce or other not so clear websites to increase their page rank on Google.
  • Stealing identity. The intention is to steal credentials or other personal information so that they can impersonate that user.
  • Phishing. In some ways overlapping with identity theft, phishing attempts to take advantage of the user's inexperience with technology to pull off a scam.
  • Spreading malware. In this case, the spam is a way to carry viruses, worms, and other malware.

How to fight spam?

First of all, you need to distinguish automated and manual spam.

In manual spam, an individual creates messages or content and sends or injects it by hand. Of course, this type of spam is expensive and slow. Even if it is still annoying, it is relatively easy to control and keep at bay.

The automated spam performs most of the spamming operations through software: from gathering the recipients to sending the messages or injecting the content.

Of course, automated spam is much more widespread and hard to fight, due to the high number of messages and content they can spread at high speed.

The typical approaches used to fight automated spam are basically the following:

  • Detecting automated injections. This approach prevents automated spamming by trying to understand if content comes from a human or a bot. The CAPTCHA is a typical technique of this approach, but, of course, it cannot be applied to messages.
  • Analyzing the content. This approach analyzes the inbound text to determine if it should be classified as spam. Spam filters perform text analysis based on a set of rules. For example, they consider the presence of a predefined set of keywords, the number of links, and so on.
  • Analyzing the source. This approach is based on the fact that, usually, automated spam attacks come from well-known servers. So, classifying the reputation of the content source can be a good way to detect possible spam.

In this article, you will not tackle the spam problem in every aspect. You will focus on a specific type of spam and will learn how to mitigate the problem by using Auth0 IP Signals. You will consider the comment spam problem, that is, the spam injected into a website via comments of an article or a product. In particular, you will analyze a simplified case of a catalog application that allows users to provide feedback and product reviews.

"Learn how to use Auth0 IP Signals to detect spammers."

Introducing Auth0 Signals

To grant the most high-level security, Auth0 uses its Anomaly Detection Engine, which analyzes actions performed on the platform. Actions like login attempt, user creation, token refresh, and so on, are inspected to determine potential risks. The checks performed by the Anomaly Detection Engine rely on a collection of dynamic data feeds called Auth0 Signals. The following animation outlines how the engine works:

 


Of course, these signals are internal services and grant security to Auth0 services. However, the IP reputation signals is offered as a standalone free service to the community. The service consists of an HTTP API that allows you to detect the reputation of the IP address of an HTTP request's source. The API checks the IP address or the domain of the request originator against several public blacklists to evaluate its reputation. It responds with data about the originator and a score indicating the reliability level of the client making the request. Having this information enables you to detect malicious activities proactively.

The Sample Application

To learn using the IP reputation signals, consider a catalog application that allows users to provide their feedback on the products. The sample application you are using here is a car catalog. The client side of this application looks as follows:

The UI of the sample application

You can browse this client online, but consider that this is just a demo application.

To build the server counterpart that accepts feedback from the users, make sure you have Node.js installed on your machine. Then, create a new folder named review-server and move into it. Now, create a package.json configuration file with the following content:

{
  "name": "review-server",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "axios": "^0.19.2",
    "cors": "^2.8.5",
    "express": "^4.17.1"
  }
}

Then, install the dependencies by typing the following command in a terminal window:

npm install

Finally, add a server.js file with the following code:

// server.js

const express = require("express");
const cors = require('cors');
const app = express();
const port = 3000;

app.use(express.json());
app.use(cors());

app.get("/", (req, res) => res.send("Welcome!"));

app.post("/review", async (req, res) => {
  const statusCode = 201;   //Created

  storeReview(req.body);

  res.status(statusCode).end();
});

app.listen(port, () => console.log(`The server is listening at http://localhost:${port}`));


function storeReview(review) {
  console.log(`Storing review ${JSON.stringify(review)}`);
}

This code defines two API endpoints using Express. The first one matches the root route (/) and simply returns a welcome message. The second one matches the /review route and accepts reviews sent via HTTP POST by the users. The logic of the controller behind the review API endpoint is trivial. It simply takes the body of the HTTP request and passes it to a storeReview() function. The storeReview() function should persist the received review somewhere, typically in a database. However, this is not relevant to the purpose of this article.

To connect the online client with this API, you need to enable CORS. In fact, the client and the server of this demo run on different domains, so your browser would refuse to make HTTP requests to this server. For this reason, you find the cors middleware in the code above.

To ensure that everything works as expected, launch the server by running the following command in a terminal window:

node server.js

By default, the server should listen on the port 3000.

Point your browser to https://great-car-catalog.netlify.com and move to the Settings tab. In the Server URI textbox, insert the URL of your server. It should be http://localhost:3000:

Catalog settings

Now you can try to add your feedback to a product and send it to the server.

This sample application allows anonymous users to send feedback to the server. A real application should only allow authenticated users to send feedback. For example, you can use Auth0 to integrate your application with authentication and authorization services out of the box.

Integrating with Auth0 IP Reputation Signal

As you may guess, this application can be an easy target for spammers. They could inject spam content in the feedback form for each product in the catalog. And they could do this several times, if the spamming is automated. To protect your application from these attacks, you can integrate the IP reputation signals.

The Auth0 Signals Dashboard

To use the API, you need an API key to pass in your request. To get this key, you have to sign up for a free account and grab it from the dashboard, as shown in the following picture:

Auth0 IP Signals dashboard

The dashboard allows you to make requests interactively to get reputation data about any IP address. You get these data in the same page, as shown below:

Auth0 IP Signals results

Processing the IP reputation score

Once you have your API key, you can integrate the Auth0 IP reputation signals in your application. So, edit the server.js file and replace its content with the following:

const express = require("express");
const cors = require('cors');
const axios = require('axios');
const app = express();
const port = 3000;

app.use(express.json());
app.use(cors());

app.get("/", (req, res) => res.send("Welcome!"));

app.post("/review", async (req, res) => {
  let statusCode = 403;

  console.log(`Review received from IP ${req.ip}: ${JSON.stringify(req.body)}`);

  const reputationScore = await getReputationScore(req.ip);

  switch (reputationScore) {
    case 0:
      storeReview(req.body); 
      statusCode = 201;   //Created
      break;

    case -1:
    case -2:
      toReviewModeration(req.body)
      statusCode = 202;   //Accepted
      break;

    default:
      statusCode = 403;   //Forbidden
      break;
  }

  res.status(statusCode).end();
});

app.listen(port, () => console.log(`The server is listening at http://localhost:${port}`));

function storeReview(review) {
  console.log(`Storing review ${JSON.stringify(review)}`);
}

function toReviewModeration(review) {
  console.log(`Review to moderate: ${JSON.stringify(review)}`);
}

async function getReputationScore(ipAddress) {
  try {
    const response = await axios.get(
        `https://signals.api.auth0.com/v2.0/ip/${ipAddress}`,
        {headers: {"X-Auth-Token": "YOUR-API-KEY"}}
      );

    return response.data.fullip.score;
  } catch (error) {
    console.error(error);
  }
}

This new version of the server code gets the reputation score for the IP address originating the HTTP request and determines the action to perform on the received feedback.

The getReputationScore() function takes the IP address and calls the IP reputation signals endpoint. It passes the API key as the value of the X-Auth-Token header. In the code above, you have to replace the YOUR-API-KEY string with your own API key. The response returned by the API endpoint contains detailed information about the IP address: IP geolocation, WHOIS information, blacklists where the IP address was found, historical activity, and so on. But here you are interested in the reputation score reported by the response.data.fullip.score property. The reputation score is a numeric value. It is determined by the checks performed by the service and can have one of the following values:

  • 0: The service found no negative data about the IP address, so it is not classified as risky.
  • -1: The service found a signal of risk for the IP address. This is the lowest level of risk.
  • -2: The service found two signals of risk, so the IP address is classified at medium risk.
  • -3: All the checks performed by the service lead to consider the IP address at high risk.

Once you get the reputation score of the calling IP address, you decide what to do with the review.

If the score is 0, the review comes from a secure source, so you pass it to the storeReview() function.

If the score of the IP address is -1 or -2, you consider the review as potential spam. So, you pass it to the toReviewModeration() function that should ideally put the feedback in a queue for manual evaluation.

When the reputation score is -3, you know that the sender is a well-known spammer, and you ignore the feedback.

Note the HTTP status codes that the controller returns. They follow the REST convention describing what happens on the server side. When the feedback is considered OK, it communicates that the feedback resource has been created on the server (201 Created). When the feedback needs a human check, it responds by telling that the server has received it, but it will be processed later on (202 Accepted). When the feedback is considered spam, it notifies the client that it is not authorized to send that feedback (403 Forbidden).

Now, after these changes, you can restart the server and try to send your feedback from the client.

"Auth0 IP Signals gives you a reputation score of any public IP address."

Simulating different sources

Using the online client with your local server does not provide you with the best experience in testing the Auth0 IP reputation signals. The resulting IP address in this combination is always the localhost (127.0.0.1, ::ffff:127.0.0.1, or ::1), so you cannot get the different results that the server can return.

You can simulate different request sources by making a few little changes to the controller of the /review route. Open the server.js file and change its content as follows:

// server.js

const express = require("express");
//... existing code ...

app.post("/review", async (req, res) => {
  let statusCode = 403;

  const ipAddress = processIpAddress(req.ip);
  console.log(`Review received from IP ${ipAddress}: ${JSON.stringify(req.body)}`);

  const reputationScore = await getReputationScore(ipAddress);

  switch (reputationScore) {
    //... existing code...
  }

  res.status(statusCode).end();
});

//... existing code...

As you can see, instead of passing the IP address of the HTTP request's originator, you are passing the IP address returned by the processIpAddress() function. You can add the definition of this function at the end of the server.js file:

// server.js

const express = require("express");

//... existing code ...

function processIpAddress(ipAddress) {
  const sampleIpAddressList = [
    "31.217.222.105",
    "45.132.104.18",
    "172.217.21.68",
    "23.222.3.98"
  ];
  let result = ipAddress;

  if (["127.0.0.1", "::ffff:127.0.0.1", "::1"].includes(ipAddress)) {
    result = sampleIpAddressList[Math.floor(Math.random() * Math.floor(4))]
  }

  return result;
}

If the IP address of the current request is localhost, the processIpAddress() function returns a public IP address. The returned address is randomly selected from the four IP addresses of the sampleIpAddressList array. These IP addresses are real addresses and have different reputation scores. For testing purposes, you can get bad reputation IP addresses from the blacklist rankings published in this page from Apility.io.

Apility.io has been acquired by Auth0 in March 2020, and its services are now part of Auth0 Signals

This way, you can simulate requests coming from these IP addresses and experience the different responses the user gets. For example, the following picture shows the message received when your feedback is awaiting moderation:

Feedback waiting for moderation

Summary

Now you have a basic but enough knowledge of Auth0 IP Signals to implement your own spam detector based on IP reputation. As you learned in this tutorial, you just need to get an API key and make an HTTP request to the API endpoint. Auth0 IP Signals provides you with other endpoints for more specific analysis. Be sure to check out the documentation for more details.

You can find the full source code of the Node server this GitHub repository.

Join the community and try out our new Slack app!

Today, we are launching the Auth0 Signals community. Go there to interact with other security professionals, get help on incident response actions, or learn more about digital abuse. In addition, Auth0 is releasing the Signals Slack app. Now teams practicing DevSecOps can embed full IP reputation information into their existing Slack workflows.