Migrating 125.000 users from Auth0 to Supabase

Over the course of a few weeks, we gradually migrated 125k users with social auth and email+password logins from Auth0 to Supabase, including a webapp, native iOS app and a single-sign-on service.

Backstory

Let me give you a little backstory first for you to understand why and how we even came to this point.

Parqet, a bootstrapped fintech startup, that allows you to visualize and analyze your wealth, has a freemium B2C (Business to Customer) business model, where the vast majority of customers do not pay a dime. As it’s heavily focused on privacy, we do not integrate ads or sell data.

For authentication, Parqet started with Auth0, a great choice to get started with. 10.000 active monthly active users for free. No need to worry about security or future-proofness, pretty much all sorts of login methods and providers supported. Auth0 worked well for a long time.

Eventually we passed the 10.000 monthly active users and upgraded to a startup plan, paying a few hundred bucks per month. We far exceeded the 10k monthly active users (pricing is based on this KPI) and had 125.000 registered users within a year and a half. One year passed and the startup plan was about to end.

Unfortunately, the pricing was waaaay out of our league and completely unsustainable for a freemium model with many non-paying users.

We made the tough decision to move to a better fitting Auth provider.

Our setup

Auth0 was tightly integrated into our system.

To summarize:

• 125k users
• Social logins (Google, Apple, Facebook) and Email+Password logins
• Single-Sign-On for our Circle community
• Two client apps (Nuxt Webapp and native iOS App)
• Using pre-built hosted login/registration page from Auth0
• Registration hooks to create users in our own database

Evaluation and requirements

Most requirements were quite obvious:

Previous users should still be able to login, so we need to support at least all previous login methods (Social auth and email+password logins).

Users should not be required to change their passwords, as we do not want 75.000 users to change their passwords.

We wanted to keep user impact as low as possible and do a rolling migration and not take the system down for a weekend or even longer — even with lots of preparation.

As one of the main reasons to migrate to a different Auth provider was reducing costs, pricing that scales with our business model was key.

I’m not going into every itty bitty detail about pros and cons of every Auth provider out there (there are a lot of options) — we did a basic evaluation of roughly 8–10 Auth providers and Supabase came out at the tippity top for us.

Why Supabase?

Supabase ticked a lot of boxes.

Fair pricing, transparent company, supported all prior social providers, supported hooks, supported email+password login, extendable, open-source, based on standard software (Postgres), fast-moving, also a startup (easier to reach out to and quicker to solve issues), no vendor lock-in. Supabase Auth is based on GoTrue — the auth system powering Netlify.

As we already had some positive prior experience with Supabase within the team, we decided to give it a go.

We contacted Supabase and they set up an initial call to talk about the migration, set up a Slack channel to get immediate feedback or help, in case of any questions.

“Why didn’t you just build your own auth?”

Every time you post anything tech related, you get a few peepz asking this.

While in some cases, it is viable to build your own (auth) system, if you have very limited resources and Auth is not a core part of your business model, you are taking a lot of resources and focus away from your actual business.

Sure, you can build your own payment gateway, auth system, CRM, mail-delivery system, subscription management, infrastructure, customer feedback tool, community software, tracking, monitoring, affiliate program.

However, you’ll likely end up maintaining all of these tools and won’t be able to advance in your core business model — that’s just my experience.

Keep in mind, it does not end with the initial implementation. It needs to be maintained, operated, updated, extended, … You end up spending a lot of time just doing that. If you have an entire team that’s just responsible for Auth in a huge enterprise company, that might be viable — not so much in a small startup.

Preparing the migration

As we planned a rolling migration and no down-times, involving two clients (webapp and native iOS app), this was quite a challenge and lots of unknowns at first.

Can we map Auth0 users to Supabase?

Can we migrate the passwords in a compatible way?

How do we get the password hashes from our Auth0 users?

How do we roll out Supabase gradually?

How do we support Single-Sign-On to our community platform?

Our main API uses Node.js with fastify. A user authenticates against our API by sending a JSON Web Token to our API. We use JWKS (JSON Web Key Sets) to verify those tokens, extract the user id and identify the requesting user.

The basic plan

We introduce feature toggles for every login method (Google, Facebook, Apple, Email+Pw Auth) to switch from Auth0 and Supabase in our clients.

We wanted to migrate Social logins first and do the Email+Password migration in one go at a later point in time. Users would be migrated on-the-fly by logging in and users that have not logged in will be migrated through a script.

When a user logs in or registers through Supabase, we try to look up the user (by email or social provider id) in our own database and match the Supabase user with a previously existing user.

By saving that information in our own database (old user + new supabase user id) and also saving the old user id in Supabase, we have a traceable user mapping in both systems.

When the user logs in again and we get a Supabase JWT, we can look up the Supabase user id and find a connected account. I’ll get into more detail on how we achieved that.

Adjusting our API

The first thing we did was extend our API to not only allow Auth0 JWTs, but also Supabase JWTs. So it wouldn’t matter, if the requesting user was logged in through Supabase or Auth0.

Supabase JWTs can be verified without doing any network requests by using the Supabase JWT secret provided in the Supabase dashboard.

Adjusting the clients

As we wanted to do a rolling migration and we had two connected clients, we wanted to introduce feature toggles for each login method and gradually migrate each method. Keep in mind, with a native iOS app, we can’t just roll out releases — we need Apple employees to be in good spirits and approve our changes.

After extending our API to support Supabase user JWTs, we integrated the Supabase SDK in both our Nuxt webapp and our native iOS app. Each login method was connected to a feature/remote toggle that allowed us to switch a login/registration method from Auth0 to Supabase.

As we could not make use of the hosted auth page from Auth0, we built our own login/registration page, which also resulted in a nice upgrade to our user experience.

Matching user accounts

Initial social auth logins and new email+password registrations would both create new Supabase users. We set up a hook that would run BEFORE inserting a user and call our own API to do a user look-up.

As Supabase is Postgres based, we can make use of Postgres features such as triggers or use the HTTP extension to make an API call.

Here’s a shortened version of the Postgres trigger:

The user matching was done within the API. Social users were the easiest to match as the Auth0 user id contains the social provider id like facebook|<facebook-user-id>and that information is also available in Supabase (within raw_user_meta_data).

Our API and clients applications were ready.

Another blocker was our community platform — Circle. We used OAuth 2.0 endpoints provided by Auth0 to support Single-Sign-On, so our users could login to our community with their credentials.

Implementing OAuth 2.0 compatible SSO

Unfortunately, Supabase does not offer any OAuth compatible endpoints to easily do such integration and Circle has no direct integration with Supabase.

With a rolling migration, we also needed a custom login as both providers needed to be supported in parallel, just like in our webapp and native app.

We ended up implementing the OAuth authorization code flow ourselves and provided a couple of endpoints — as this is standardized, it was okay to implement — took roughly a day or two to implement and test through. Circle was nice enough to provide us with a separate Enterprise account for testing purposes.

Writing migration scripts

We knew that not all of our 125k registered users would login within a short amount of time, so we knew we had to migrate the remaining users.

To do so, we wrote scripts that would take the exported Auth0 users (exported using the export “User Import/Export” extension), check for unmigrated users, map them to the Supabase data formats and tables and insert them directly.

Besides some test iterations, this was fairly straight-forward. You can find some of our migration scripts on Github to get an idea on how to map Auth0 users to Supabase.

Duplicate email addresses

While Supabase allows linking accounts, like having an email+password login and adding Google social login (happens automatically when the email matches), Supabase does not allow having multiple accounts with the same email address — while Auth0 does.

We had a few thousand cases of multiple user accounts with the same email address. Most were pretty obvious to solve and seemed like accidental logins — i.e. an email+password user that had a bunch of activity and a social user with the same email address that only logged in once and had no activity in our system.

We deleted the inactive users and were left with roughly 100 cases that were impossible to solve automatically — we postponed solving those cases, though.

Sending a welcome mail

While Supabase has a few built-in and configurable mail templates, there is no email triggered upon user registration to welcome a user. As we already had a Supabase hook that does an API call and notifies our API about a new user being created, we simply used our existing Mandrill integration to send a transactional mail.

Migrating user passwords

We were worried that users may have to change their password due to hash incompatibility.

Luckily, both Auth0 and Supabase use bcrypt as password-hashing algorithm. We requested an export for password hashes from Auth0 and were able to verify that the hashes also work in Supabase, without any adjustments.

Migration

After preparing the feature toggles, adjusting the API, building user matching, the Postgres trigger, adjusting both clients and releasing all of that and A LOT of testing, we were ready to pull the trigger.

We first rolled out the new login/registration page, replacing the hosted page and rolled out all other changes in a backwards compatible way (the API, webapp and native app).

Afterwards, we switched our community Single-Sign-On from the Auth0 integration to our own endpoints that supported both Auth0 and Supabase, in a backwards compatible way.

We migrated our dev system first and were able to try out a few things during the migration — all went well.

Next, we started to toggle the first social auth provider, meaning, all new logins and registrations would go through Supabase — logins from previously existing users would be migrated on-the-fly and connected to their old account.

We observed it for a bit and toggled the other remaining social providers. After a few hours, we migrated all remaining social auth users that did not login through our migration script. This was all during the course of a day.

A cool “trick” to forcefully log out all remaining Auth0 users logged in through social auth, was to disable the social connections in the Auth0 management interface. When trying to refresh the token after a couple of minutes, the refresh failed, forcefully logged out the user and required the user to log in again and be migrated on-the-fly.

This surely did not go through without any issues — we still had some cases we didn’t consider before and had to fix some things on the fly — all in all it went well, though.

Now we had to wait for the final export of password hashes from Auth0 — unfortunately we couldn’t pick a time. We waited roughly 1 1/2 weeks to get the export and it finally came at 2am in the morning on the Saturday of the Easter weekend — great time to upset your family/partner because you spend time migrating users by the way ;)

We migrated all remaining users and realized there were 1600 users missing in the export — it was already too late and we had migrated > 72.000 users.

As this was the Easter weekend, we wouldn’t be able to get the export from Auth0 in time. We generated individual passwords, contacted those users via email and told them to manually use the password reset flow to set a new password.

We spent the upcoming days improving the auth system (frequent logouts, password reset flow, some cases where token refreshes did not work properly, …) and solving a few individual cases for users with duplicate email addresses and linked accounts (see pitfalls).

Pitfalls

Apple app approvals

We integrated Supabase as secondary Auth provider and implemented remote toggles (using Firebase).

After publishing the release, ready to migrate, the Apple employee was not in good spirits and decided that we need to implement something completely unrelated, some link to restore In-App-Purchases before allowing the release.

Anyway — this blocked us for a couple of days and delayed our entire migration.

Duplicate emails

As described above, Supabase supports multiple identities for a single email, but not multiple accounts with the same email.

This was annoying and time-consuming to solve as some cases had to be looked at manually and even after the migration we had to re-attach/reconnect some users.

Linked Accounts

As if duplicate emails aren’t complex enough, we had the Auth0 link extension active for a while, which allowed others to link multiple login methods/providers to a single account.

We also had to take this into account when matching users and migrating, so if an Auth0 had multiple identities, we also had to create multiple identities in Supabase. As we realized this too late, this caused quite a few issues with falsely matched user accounts. Luckily, only a small percentage of users was affected.

In hindsight, we should have taken this into account from the very beginning, we just overlooked it.

Getting Password hashes

With Supabase, you get full access to the underlying Postgres database. Thus, you can also just get an export of all password hashes.

With Auth0, you have no way to access the password hashes besides creating a support ticket. We went through a few levels of support to get the password hashes and it took about a week in total.

We needed the export twice — once for initially testing the migration and a second time for doing the final migration. Auth0 is unable to time those export and they work in a different timezone — so the final export came at 02:00 AM in the morning on a Saturday and forced us to migrate that day (Easter weekend).

Resumé

Phew. I have to admit, I am glad the migration is done. It was a fun and exhausting challenge.

As we had a pretty complex scenario, we ran into a bunch of mind-boggling and time-consuming edge-cases. After a few headaches and sweaty sessions, we were able to migrate all our users to Supabase, though.

We greatly reduced costs, have a reliable partner that scales with our business model, a stable authentication system and sail smoothly.

One annoying side effect we also solved with Supabase is occasional rate limits exceeding. As Auth0 has global rate limits, when we ran a small script that queried the Auth0 management API, we quickly ran into a global rate limit that would not even let our own users log out of our system. No more, with Supabase.

We are more than happy with the new auth provider.

The end

I didn’t go into every nuance and detail, as this would likely fill an entire book.

A migration like this is highly individual, but I do hope you got something useful out of this. If you actually read through, I owe you one — thanks for sticking with me.

Special thanks to everyone involved from the Parqet and Supabase team and hustling through. 💚