What Could Possibly Go Wrong? - Redeeming Promo Codes

Laravel PHP Concurrency Race Conditions Backend

In today's experiment, we will see what could possibly go wrong when redeeming promo codes. Let's start with the functional requirements:

  • We have a limited number of promo codes available.
  • Users can request a code by entering their email.
  • Once a code is assigned, the user receives it by email.

From these requirements, we can define two business rules:

  • Each email can only redeem one code.
  • We can't redeem more codes than we have available.

For each implementation, we will check if these rules are being respected. We are going to use k6 for this.

k6 is a load testing tool that lets us simulate multiple users making HTTP requests. Instead of manually clicking a button a hundred times, we can tell k6 how many users and requests we want to simulate.

To collect the results, I used a small class with counters. Here's what we're measuring:

  • Accepted: Number of requests that successfully redeemed a code, according to the API response.
  • Users with multiple redemptions: Number of email addresses that received more than one successful redemption response.
  • Unique constraint failures: Number of requests rejected because of a database unique constraint violation.

Let's go!

Case 1: The basic implementation

Let's start with a basic implementation. The flow is simple:

  • Check if the email has already redeemed a code.
  • Find an available code.
  • Assign the code to the email.
  • Send the code by email.

Just that.

public function __invoke(Request $request): JsonResponse
{
    $validated = $request->validate(array(
        'email' => 'required|email',
    ));

    $email = $validated['email'];

    if (RedeemCode::query()->where('email', $email)->exists()) {
        return response()->json(array(
            'message' => 'A code has already been redeemed for this email.',
        ), 422);
    }

    $redeem_code = RedeemCode::query()->whereNull('email')->first();

    if ($redeem_code === null) {
        return response()->json(array(
            'message' => 'No redeem codes are available.',
        ), 409);
    }

    $redeem_code->email = $email;
    $redeem_code->redeemed_at = now();
    $redeem_code->save();

    SimulatedEmailJob::dispatch($email, $redeem_code->code);

    return response()->json(array(
        'message' => 'Code redeemed.',
        'code' => $redeem_code->code,
    ));
}

Easy peasy. It should work, right?

Let's put it to the test. We have 50 available codes and 100 users competing to get one. Just like a real promotion where everyone rushes to claim a code.

We can simulate this with k6. Let's see how:


import http from 'k6/http';
import exec from 'k6/execution';


export const options = {
    scenarios: {
        redeem: {
            executor: 'shared-iterations',
            vus: 100,
            iterations: 100,
        },
    },
};

const unique_emails = 100;

export default function () {
    const user_id = exec.scenario.iterationInTest % unique_emails;
    const email = `user${user_id}@example.com`;
    const payload = JSON.stringify({ email: email });

    http.batch([
        ['POST', redeem_url, payload, request_params],
        ['POST', redeem_url, payload, request_params],
        ['POST', redeem_url, payload, request_params],
    ]);
}

This is the relevant part of the script. redeem_url and request_params are defined elsewhere in the test.

We are using 100 virtual users and 100 iterations, with a different email for each iteration. But instead of sending just one request, we send three requests for the same email concurrently. Because, well, who hasn't clicked a button a million times when trying to buy something?

That gives us 300 HTTP requests competing for just 50 codes. The shared-iterations executor distributes the iterations between the virtual users, so they won't necessarily all start at exactly the same moment.

The test takes about 5 seconds. Before each run, I reset the database so we always start with the same 50 available codes. I repeated the test five times.

RunAcceptedUsers with multiple redemptions
17219
27820
37417
47619
58627

If we have only 50 codes, how the hell are we accepting more than we have? We are checking everything in the code. What is going on here? We only have two business rules and we couldn't comply with either of them.

Well... let's talk about concurrency. Our code works but not when there is concurrency involved. What happens if two requests enter at the same time? Let's see:

Request ARequest B
Checks for an existing redemption: none.
Checks for an existing redemption: none.
Finds available code PROMO-001.
Finds the same available code PROMO-001.
Assigns PROMO-001 to Alice.
Assigns PROMO-001 to Bob, overwriting Alice.
Returns success.Returns success.

Both requests saw the code as available before either one saved the assignment. Both return success, even though there is only one row in the database and the second update overwrites the first. That's a race condition: the result depends on the order in which concurrent requests execute.

And it doesn't even need to be two different users. If Alice clicks twice, both requests can check that she doesn't have a code yet and end up assigning two different codes to her.

So, how can we fix this? We need somehow to prevent two requests from getting the same available code.

Case 2: Lock me this

We need somehow to block a request when someone is getting a code. We can achieve that with a database transaction and a database lock.

public function __invoke(Request $request): JsonResponse
{
    $validated = $request->validate(array(
        'email' => 'required|email',
    ));

    $email = $validated['email'];

    if (RedeemCode::query()->where('email', $email)->exists()) {
        return response()->json(array(
            'message' => 'A code has already been redeemed for this email.',
        ), 422);
    }

    $redeem_code = DB::transaction(function () use ($email) {
        $redeem_code = RedeemCode::query()
            ->whereNull('email')
            ->lockForUpdate()
            ->first();

        if ($redeem_code === null) {
            return null;
        }

        $redeem_code->email = $email;
        $redeem_code->redeemed_at = now();
        $redeem_code->save();

        return $redeem_code;
    });

    if ($redeem_code === null) {
        return response()->json(array(
            'message' => 'No redeem codes are available.',
        ), 409);
    }

    SimulatedEmailJob::dispatch($email, $redeem_code->code);

    return response()->json(array(
        'message' => 'Code redeemed.',
        'code' => $redeem_code->code,
    ));
}

The important change is lockForUpdate() inside DB::transaction(). This tells MySQL to lock the selected records while the transaction is open. Another transaction trying to lock the same records may need to wait until the first one finishes. The locks are released when the transaction commits or rolls back.

This prevents two requests from successfully reserving the same available code. If there are no codes left, we return a 409 response. If we get a code, we assign it and dispatch the email job after the transaction.

With that, it should be working. Let's execute our k6 script and see.

RunAcceptedUsers with multiple redemptions
15016
25017
35012
45014
55015

Great! We fixed the stock problem. Exactly 50 codes were redeemed in every run.

But wait... why are some users still getting multiple codes?

In Case 1 we talked about race conditions and concurrency. We protected the available codes, but what about checking if that user already redeemed a code? That check is still outside the transaction.

If several requests enter at the same time, they can all see that the email doesn't have a code yet. Then they wait their turn for an available code. The database lock protects the code row, but it doesn't automatically make that earlier email check valid again.

How can we be sure that just one email gets one code? Maybe we need to add something that should be there from the very beginning: a unique constraint on the email column.

Case 3: Unique code for a unique email

By adding a unique index on the email column, the database is going to protect us if we try to assign another code to an email that already has one.

$table->unique('email');

The email column is nullable because unused codes don't have an owner yet. MySQL allows multiple NULL values in a unique index, so we can still have all 50 unassigned codes. But once an email is assigned, that same email can't be assigned to another row.

We can wrap the transaction with a try/catch to capture the exception and return the proper response to the user.

public function __invoke(Request $request): JsonResponse
{
    $validated = $request->validate(array(
        'email' => 'required|email',
    ));

    $email = $validated['email'];

    if (RedeemCodeUnique::query()->where('email', $email)->exists()) {
        return response()->json(array(
            'message' => 'A code has already been redeemed for this email.',
        ), 422);
    }

    try {
        $redeem_code = DB::transaction(function () use ($email) {
            $redeem_code = RedeemCodeUnique::query()
                ->whereNull('email')
                ->lockForUpdate()
                ->first();

            if ($redeem_code === null) {
                return null;
            }

            $redeem_code->email = $email;
            $redeem_code->redeemed_at = now();
            $redeem_code->save();

            return $redeem_code;
        });
    } catch (UniqueConstraintViolationException $e) {
        return response()->json(array(
            'message' => 'A code has already been redeemed for this email.',
        ), 422);
    }

    if ($redeem_code === null) {
        return response()->json(array(
            'message' => 'No redeem codes are available.',
        ), 409);
    }

    SimulatedEmailJob::dispatch($email, $redeem_code->code);

    return response()->json(array(
        'message' => 'Code redeemed.',
        'code' => $redeem_code->code,
    ));
}

The initial email check is still there, but now it isn't our only protection. If two requests pass that check at the same time, the unique constraint prevents the second one from assigning another code to the same email. Laravel rolls back the transaction, catches the database exception, and the API responds with 422.

Let's see what happens now:

RunAcceptedUsers with multiple redemptionsUnique constraint failures
150049
250048
350063
450026
550025

It seems like now we achieved the two business rules:

  • A user can only get one code.
  • We can't give more codes than available.

And look at those unique constraint failures. Those are requests that passed the initial check but tried to assign an email that another request had already redeemed. Without the database constraint, we would be relying on the timing of our PHP code again.

The number of accepted redemptions is still 50, but this time no user received multiple codes. That's what we wanted.

Conclusion

We started with a simple implementation that looked perfectly fine. We checked if the email already had a code, checked if there were codes available, and assigned one. What could possibly go wrong?

Apparently, quite a lot. With just 100 users competing for 50 codes, we managed to accept up to 86 redemptions. Even after adding transactions and locks, some users were still getting multiple codes.

Adding a unique constraint finally made both business rules hold in our tests. And the funny thing is that the original implementation probably would have worked perfectly fine if we had tested it manually, one request at a time.

I wonder how many times I've written code like this without ever noticing the problem.