To significantly improve the platform’s ability to work with files and add support for large files, we intend to introduce a number of changes:
- New Data Type: Blob
We will introduce a new data type called Blob—a special structure designed to handle binary data. Blob support will allow us to work with files of almost any size without explicitly manipulating bytes. Instead, we will use pointers/references to files, eliminating the need to load entire files into memory. You will still be able to read part or all of a blob into an array of bytes or create a blob from bytes, strings, or text. - Deprecating File ID
We will deprecate the artificial File ID data type in favor of a regular string. The system model File will have an ID of the new string type, fully compatible with previous IDs. Additionally, the field currently using an array of bytes will be replaced with a blob in this model. - UUIDv7 for New File IDs
All new file IDs will be generated using the UUIDv7 format, which significantly improves performance in projects with a large number of files. Existing projects will use UUIDs for new file IDs but will maintain string data types under the hood for compatibility. New projects will natively use the UUID type in PostgreSQL for maximum performance. Migration to the UUID database type for existing projects will be handled case by case.
Modern browsers handle blobs efficiently with numerous built-in optimizations, allowing us to rely on their implementations and keep our codebase small and robust. In Go, Swift, and Kotlin, we will use native ByteArrays and ArrayBuffers with abstraction levels ensuring blobs are only loaded into memory when needed.
While these changes may cause some inconvenience for existing developers, they are necessary to resolve current file-handling issues. We expect to release planned changes in Q1-2025.
We welcome your feedback and suggestions on how to improve file handling further.