Related: file format
Question
How do I want to store my data?
It can add complexity for querying if not designed properly:
- access patterntodo
- data modality
- serializing models
Data Modality
Text
Images
Audio etc.
Tabular
Model serialization formats
| Format | Pros | Cons |
|---|---|---|
pickle | Ideal for loading Python objects | Insecure to load untrusted model files |
joblib 1 | Ideal for loading numpy arrays | Insecure to load untrusted model files |
| Tensorflow model formats | ||
pt | Specific to PyTorch | |
[[ONNX|onnx]] | - A framework-agnostic format2 - Allow execution in different environments, languages and hardware | - Limited to operations supported by ONNX |