Snaplet Snapshot with Google Cloud Storage

Tobias Törnros

August 29, 2024

Startups are hard. One of our favorite startups, Snaplet, is shutting down. Despite that, they built an amazing team and some incredible products.

Peter Pistorius, Snaplet’s founder, shared:

“I built Snaplet because I believe developers write better software when they have access to production-like data. Although the company is closing, my belief remains strong, so we are open-sourcing the tools we’ve built.”

We want to express our heartfelt thanks to Peter and the entire Snaplet team for their innovative contributions to the developer community. Their work has significantly improved how we handle database snapshots and generate realistic test data. We’re also grateful to Supabase for stepping in to help maintain these tools, ensuring they remain available for developers to use and build upon.

Open sourcing Snaplet

Snaplet is releasing three main tools under the MIT license:

Copycat: Generates deterministic fake data.

Snaplet Seed: Generates realistic synthetic data based on a database schema.

Snapshot: Captures, transforms, and restores snapshots of your database.

Migrating Snaplet Snapshots to Google Cloud Storage

With Snaplet’s cloud service closing, many teams need an alternative for storing and managing their snapshots. Google Cloud Storage (GCS) offers a reliable solution. Here’s how you can migrate:

1. Set Up Google Cloud Storage

Create a GCS bucket for your snapshots:

gsutil mb gs://your-snaplet-bucket

2. Update Your Snapshot Creation Workflow

Here’s a script to create and upload snapshots to GCS:

#!/bin/bash

BUCKET_NAME="your-snaplet-bucket"

# Create snapshot
pnpm snaplet snapshot capture

# Upload to GCS
gsutil -m rsync -r packages/seed/.snaplet/snapshots gs://$BUCKET_NAME/snapshots/

# Clean old snapshots (keep the 2 most recent)
SNAPSHOTS=$(gsutil ls gs://$BUCKET_NAME/snapshots/ | sort -r)
echo "$SNAPSHOTS" | tail -n +3 | while read -r snapshot; do
    gsutil -m rm -r "$snapshot"
done

3. Create a Download Script

For easy snapshot downloads:

#!/bin/bash

BUCKET_NAME="your-snaplet-bucket"
SNAPSHOT_DIR="packages/seed/.snaplet/snapshots"

mkdir -p "$SNAPSHOT_DIR"
rm -rf "$SNAPSHOT_DIR"/*

gsutil -m rsync -r gs://$BUCKET_NAME/snapshots/ "$SNAPSHOT_DIR"

echo "Snapshots downloaded to $SNAPSHOT_DIR"

‍

4. Update Your package.json

Add these scripts to your package.json:

{
  "devDependencies": {
     "@snaplet/snapshot": "^0.93.2",
  },
  "scripts": {
    "download-snapshots": "bash ./scripts/download-snapshots.sh",
    "setup-db": "pnpm run download-snapshots && pnpm exec snapshot snapshot restore"
  }
}

5. Automate with CI/CD

Consider automating snapshot creation and downloads using CI/CD tools. This can be set up to run on a schedule, ensuring your team always has access to up-to-date snapshots.

Security Considerations

Use GCP’s IAM for access management
Encrypt sensitive data before uploading
Regularly audit access logs
Ensure minimal permissions for service accounts
Rotate service account keys regularly

Conclusion

While Snaplet’s closure is unfortunate, their decision to open-source their tools ensures their work will continue to benefit the developer community. By migrating snapshots to GCS and utilizing these open-source tools, teams can maintain an efficient workflow.

As we adapt to these changes, let’s once again thank Peter Pistorius, the Snaplet team, and Supabase for their valuable contributions and ongoing support. Their commitment to the developer community is truly commendable.

Remember to adjust the provided scripts to fit your project’s needs. With these tools and processes, you can continue leveraging the power of database snapshots in your development workflow.

‍