Building your own URL shortener with Typescript and Redis

Have you ever wonderered how URL shorteners work? I thought today it would be fun to build a simple implementation, and discuss some of the considerations that go into building a production-ready URL shortener. I'll be building out a simple prototype using JavaScript objects, and then I'll show you how to swap out the in-memory store for Redis.

What this won't be is a fully fleshed out, production-ready solution, with all the edge cases an scaling considerations addressed. However, it should give you a good starting point to build out your own URL shortener.

What is a shortened URL?

You've probably seen shortened URLs before, they look something like this: https://bit.ly/3j4i2l. The idea is that you can take a long URL, and shorten it to something much more manageable. You can imagine when this might be useful, for example, when sharing a link on Twitter (ok, ok, X) or some other platform, posting the full URL might not be practical or asthetically pleasing.

But how does this actually happen? How do you take a long URL and shorten it to something much more manageable? The answer is actually quite simple, you just need to create a mapping between the long URL and the short URL. When a user requests the short URL, you look up the long URL in the mapping, and then redirect the user to the long URL.

A really naive solution

We'll start by building a very basic implementation, using a counter to generate the short URL. This is a very naive solution, but it will give us a good starting point. Consider the following code:

type ShortenerService = {
  shorten(url: string): string
  redirect(shortUrl: string): string
}

function shortenerService(): ShortenerService {
  let counter = 0
  const urlMap = {}

  return {
    shorten(url: string) {
      const shortUrl = `https://our-shortener.url/${counter}`
      urlMap[shortUrl] = url
      counter++
      return shortUrl
    },

    redirect(shortUrl: string) {
      return urlMap[shortUrl]
    },
  }
}

const shortener = shortenerService()

shortener.shorten('https://www.nytimes.com?article=1234')
// => https://our-shortener.url/0

shortener.shorten('https://www.nytimes.com?article=5678')
// => https://our-shortener.url/1

shortener.redirect('https://our-shortener.url/0')
// => https://www.nytimes.com?article=1234

shortener.redirect('https://our-shortener.url/1')
// => https://www.nytimes.com?article=5678

Despite its simplicity, this implementation is actually quite powerful. It allows us to take a URL of abitrary length, and shorten it to a fixed length URL. We can then use the shortened URL to redirect the user to the original URL. This is the basic idea behind a URL shortener. However, there are a few problems with this implementation.

First, let's think about that counter. What happens when our service gets really big, and we have billions of URLs? Our urls suddenly aren't so short anymore. Also, using the counter, our urls are very easy to guess. If I make a short URL and see that the mapping code is 4555, I can go see what all the other 4554 URLs are. This is not ideal. To be clear, you shouldn't use a URL shortener if you have a secure URL that you don't want to share publicly, but even so, we can do better.

Another concern is that the object of URLs is stored in local memory. This is going to be a real problem if we actually hope to use our service. Just imagine what would happen if our service went down. That object store would be completely wiped out, and all of our shortened URLs would be lost. This is not ideal. Furthermore, if we wanted to scale our service and run multiple instances of this code, each one is going to have its own object store, and we would have to figure out how to keep them in sync. So we're going to need to keep our URL map somewhere more persistent.

Improving our short code

There are a couple obvious problems with our current short codes. First, as mentioned, the incrementing short codes are very limiting. Second, it's easy to guess what other short codes are. And finally, if you consider how they're being made, they're not a standard length. That last one might not seem like a big deal, but our shortened URLs will look funny if they all have different lengths.

Instead of using a counter, we can use a hashing function to generate a nearly unique value for each URL. This will give us a much larger pool of possible short codes, and make it much harder to guess what other short codes are. There are any number of ways you could go about this, but for our purposes, we'll use a SHA-256 hash, and convert the result to a base36 string.

const crypto = require('crypto')

// Function to hash a value using SHA-256 and encode in base36
function makeBase36Hash(input) {
  // Step 1: Create a SHA-256 hash of the input
  const hash = crypto.createHash('sha256')

  // Step 2: Update the hash with the input data
  hash.update(input)

  // Step 3: Get the hexadecimal digest of the hash
  const hexHash = hash.digest('hex')

  // Step 4: Convert the hexadecimal hash to an integer
  const intHash = BigInt(`0x${hexHash}`)

  // Step 5: Convert the integer hash to a base36 string
  const base36Hash = intHash.toString(36)

  return base36Hash
}

makeBase36Hash('https://www.nytimes.com?article=1234')
// => '40l863gxbi29wfqltti7uarynn0cgb8xcghq7fpzx4uksosmqy'

Now you might say "Wait, that's not short at all!" And you'd be right. But we don't really need to worry about the length of the hash. We can just take the first 7 characters of the hash, and use that as our short code. This will give us a pool of 78 billion possible short codes, which should be more than enough for our purposes.

So, let's update our shortenerService to use this new hashing function:

import crypto from 'crypto'

type ShortenerService = {
  shorten(url: string): string
  redirect(shortUrl: string): string
}

function shortenerService(): ShortenerService {
  const urlMap = {}

  return {
    shorten(url: string) {
      const shortCode = makeBase36Hash(url).slice(0, 7)
      const shortUrl = `https://our-shortener.url/${shortCode}`
      urlMap[shortUrl] = url
      return shortUrl
    },

    redirect(shortUrl: string) {
      return urlMap[shortUrl]
    },
  }
}

function makeBase36Hash(input: string): string {
  const hash = crypto.createHash('sha256')
  hash.update(input)
  const hexHash = hash.digest('hex')
  const intHash = BigInt(`0x${hexHash}`)
  const base36Hash = intHash.toString(36)

  return base36Hash
}

const shortener = shortenerService()

shortener.shorten('https://www.nytimes.com?article=1234')
// => https://our-shortener.url/40l863g

shortener.redirect('https://our-shortener.url/40l863g')
// => https://www.nytimes.com?article=1234

Now, no matter how long the URL is, we can generate a short code that is always 7 characters long. This is a big improvement over our previous implementation. There's still some edge cases that I'll mention later, but this is a good start. However, we still have the problem of storing our URL map in local memory. Let's fix that next.

Using Redis as a URL store

As I mentioned earlier, storing our URL map in local memory is not ideal. If our service goes down, or even just restarts, we lose all of our shortened URLs. That simply won't do. We need to store our URL map somewhere more persistent. There are many options for this, but as I mentioned in the title, we'll be using Redis.

Redis is an in-memory key/value database that's used for many differnt caching and storage use cases. It's very fast, and it's very easy to use. We'll be using the redis npm package to interact with Redis. If you don't have Redis installed, you can run it in a Docker container like so:

docker run --name redis -p 6379:6379 -d redis

Note that you'll need to have a docker daemon running on your machine for this to work. You can download Docker Desktop from the Docker website. There are other ways to install Docker (as well as Redis), but this will get you up and running quickly.

Redis has a wide variety of data types it can handle, but for our purposes, we will just be storing key/value pairs. We'll store the short code as the key, and the long URL as the value. In this way, you really can think of Redis as a giant JavaScript object. And because it's in memory, instead of on hard storage, it's very fast.

Here's how we can update our shortenerService to use Redis:

import crypto from 'crypto'
import { createClient } from 'redis'

type ShortenerService = {
  shorten(url: string): Promise<string>
  redirect(shortUrl: string): Promise<string | null>
}

async function shortenerService(): Promise<ShortenerService> {
  const client = await createClient()
    .on('error', (err) => console.error('Redis Client error:', err))
    .connect()

  return {
    async shorten(url: string) {
      const shortCode = makeBase36Hash(url).slice(0, 7)
      await client.set(shortCode, url)
      return `https://our-shortener.url/${shortCode}`
    },

    async redirect(shortCode: string) {
      const url = await client.get(shortCode)
      if (!url) {
        throw new Error('URL not found for the given short code')
      }
      return url
    },
  }
}

function makeBase36Hash(input: string): string {
  const hash = crypto.createHash('sha256')
  hash.update(input)
  const hexHash = hash.digest('hex')
  const intHash = BigInt(`0x${hexHash}`)
  const base36Hash = intHash.toString(36)

  return base36Hash
}

;(async () => {
  const shortener = await shortenerService()
  const shortUrl = await shortener.shorten(
    'https://www.nytimes.com?article=1234',
  )
  console.log(shortUrl) // => https://our-shortener.url/40l863g

  const originalUrl = await shortener.redirect('40l863g')
  console.log(originalUrl) // => https://www.nytimes.com?article=1234
})()

The biggest change here that instead of accessing a local object, we're calling out to redis to set (client.set) and get (client.get) the values. This is a big improvement, as now our URL map is stored in a persistent data store, and we can scale our service by running multiple instances of our code.

Considering some gotchas and edge cases

Before you take this code and run with it, there are a few things you should take into consideration. As I mentioned earlier, using the 7 character hash gives us 78 billion possible short codes. This is a lot, but it's not infinite, and collisions are absolutely possible. One way you could address this is by checking for the collision before setting the short code in Redis. Should there be a collision, you could generate a new hash with a slightly different input, and try again. Additionally, you could work with a longer hash. Even just 8 characters would give you 2.8 trillion possible short codes.

Another thing to consider is where our data is stored, and how durable it is. Currently, we're running Redis locally, alongside our code. This is fine for development, but in production, you would want to run Redis in a separate instance, or even a separate data center. As a memory store, Redis is very fast, but the tradeoff is that it's not durable. At a minimum, we'd want to have our Redis instance set up with some kind of replication. Better yet, we could back our Redis instance with a more durable data store, such as DynamoDB or S3.

Finally, recognize that this code is not production ready. It's assuming that errors won't happen, and that Redis will always be available. In a production system, you would want to handle errors more gracefully. For example, if Redis is down, you might want to return a 503 status code to the user, rather than throwing an error. You might also want to add some kind of rate limiting, to prevent abuse of your service. Also, you should consider how you're going to handle the case where a user requests a short URL that doesn't exist. In our current implementation, we throw an error. This is fine for development, but in production, you would want to handle this more gracefully. You could redirect the user to a 404 page, or even redirect them to a default URL.

Conclusion

So there you have it, a simple URL shortener built with JavaScript objects and converted to use Redis. I hope you found this article interesting, and that it gave you some ideas for building your own URL shortener. There are many ways you could expand on this code, and many considerations you would need to take into account if you were to build a production-ready URL shortener. I hope this article has given you a good starting point, and that you have fun building your own URL shortener. If you have any questions or comments, please feel free to shoot me a message. Thanks for reading!