Mixin for SQLAlchemy-models serialization without pain
Mixin for SQLAlchemy models serialization without pain.
If you want to serialize SQLAlchemy model instances with only one line of code,
and tools like marshmallow
seems to be redundant and too complex for such a simple task,
this mixin definitely suits you.
Contents
pip install SQLAlchemy-serializer
If you want SQLAlchemy model to become serializable, add SerializerMixin in class definition:
from sqlalchemy_serializer import SerializerMixin
class SomeModel(db.Model, SerializerMixin):
...
This mixin adds .to_dict() method to model instances. So now you can do something like this:
item = SomeModel.query.filter(...).one()
result = item.to_dict()
You get values of all SQLAlchemy fields in the result
var, even nested relationships
In order to change the default output you shuld pass tuple of fieldnames as an argument
rules
argumentonly
argumentIf you want to exclude a few fields for this exact item:
result = item.to_dict(rules=('-somefield', '-some_relation.nested_one.another_nested_one'))
If you want to add a field which is not defined as an SQLAlchemy field:
class SomeModel(db.Model, SerializerMixin):
non_sql_field = 123
def method(self):
return anything
result = item.to_dict(rules=('non_sql_field', 'method'))
Note that method or a function should have no arguments except self, in order to let serializer call it without hesitations.
If you want to get exact fields:
result = item.to_dict(only=('non_sql_field', 'method', 'somefield'))
Note that if somefield is an SQLAlchemy instance, you get all it's serializable fields. So if you want to get only some of them, you should define it like below:
result = item.to_dict(only=('non_sql_field', 'method', 'somefield.id', 'somefield.etc'))
You can use negative rules in only
param too.
So item.to_dict(only=('somefield', -'somefield.id'))
will return somefiled
without id
. See Negative rules in ONLY section
If you want to define schema for all instances of particular SQLAlchemy model, add serialize properties to model definition:
class SomeModel(db.Model, SerializerMixin):
serialize_only = ('somefield.id',)
serialize_rules = ()
...
somefield = db.relationship('AnotherModel')
result = item.to_dict()
So the result
in this case will be {'somefield': [{'id': some_id}]}
serialize_only and serialize_rules work the same way as to_dict's arguments
For more examples see tests
class FlatModel(db.Model, SerializerMixin):
"""
to_dict() of all instances of this model now returns only following two fields
"""
serialize_only = ('non_sqlalchemy_field', 'id')
serialize_rules = ()
id = db.Column(db.Integer, primary_key=True)
string = db.Column(db.String(256), default='Some string!')
time = db.Column(db.DateTime, default=datetime.utcnow())
date = db.Column(db.Date, default=datetime.utcnow())
boolean = db.Column(db.Boolean, default=True)
boolean2 = db.Column(db.Boolean, default=False)
null = db.Column(db.String)
non_sqlalchemy_dict = dict(qwerty=123)
class ComplexModel(db.Model, SerializerMixin):
"""
Schema is not defined so
we will get all SQLAlchemy attributes of the instance by default
without `non_sqlalchemy_list`
"""
id = db.Column(db.Integer, primary_key=True)
string = db.Column(db.String(256), default='Some string!')
boolean = db.Column(db.Boolean, default=True)
null = db.Column(db.String)
flat_id = db.Column(db.ForeignKey('test_flat_model.id'))
rel = db.relationship('FlatModel')
non_sqlalchemy_list = [dict(a=12, b=10), dict(a=123, b=12)]
item = ComplexModel.query.first()
# Now by default the result looks like this:
item.to_dict()
dict(
id=1,
string='Some string!',
boolean=True,
null=None,
flat_id=1,
rel=[dict(
id=1,
non_sqlalchemy_dict=dict(qwerty=123)
)]
# Extend schema
item.to_dict(rules=('-id', '-rel.id', 'rel.string', 'non_sqlalchemy_list'))
dict(
string='Some string!',
boolean=True,
null=None,
flat_id=1,
non_sqlalchemy_list=[dict(a=12, b=10), dict(a=123, b=12)],
rel=dict(
string='Some string!',
non_sqlalchemy_dict=dict(qwerty=123)
)
)
# Exclusive schema
item.to_dict(only=('id', 'flat_id', 'rel.id', 'non_sqlalchemy_list.a'))
dict(
id=1,
flat_id=1,
non_sqlalchemy_list=[dict(a=12), dict(a=123)],
rel=dict(
id=1
)
)
If your models have references to each other or you work with large trees you need to specify where the serialization should stop.
item.to_dict('-children.children')
In this case only the first level of children
will be included
See Max recursion
If you want to change datetime/date/time/decimal format in one model you can specify it like below:
from sqlalchemy_serializer import SerializerMixin
class SomeModel(db.Model, SerializerMixin):
__tablename__ = 'custom_table_name'
date_format = '%s' # Unixtimestamp (seconds)
datetime_format = '%Y %b %d %H:%M:%S.%f'
time_format = '%H:%M.%f'
decimal_format = '{:0>10.3}'
id = sa.Column(sa.Integer, primary_key=True)
date = sa.Column(sa.Date)
datetime = sa.Column(sa.DateTime)
time = sa.Column(sa.Time)
money = Decimal('12.123') # same result with sa.Float(asdecimal=True, ...)
If you want to change format in every model, you should write
your own mixin class inherited from SerializerMixin
:
from sqlalchemy_serializer import SerializerMixin
class CustomSerializerMixin(SerializerMixin):
date_format = '%s' # Unixtimestamp (seconds)
datetime_format = '%Y %b %d %H:%M:%S.%f'
time_format = '%H:%M.%f'
decimal_format = '{:0>10.3}'
And later use it as usual:
from decimal import Decimal
import sqlalchemy as sa
from some.lib.package import CustomSerializerMixin
class CustomSerializerModel(db.Model, CustomSerializerMixin):
__tablename__ = 'custom_table_name'
id = sa.Column(sa.Integer, primary_key=True)
date = sa.Column(sa.Date)
datetime = sa.Column(sa.DateTime)
time = sa.Column(sa.Time)
money = Decimal('12.123') # same result with sa.Float(asdecimal=True, ...)
All date/time/datetime/decimal
fields will be serialized using your custom formats.
format
syntax%s
,datetime
formats you can find in docsBy default the library can serialize the following types:
- int
- str
- float
- bytes
- bool
- type(None)
- uuid.UUID
- time
- datetime
- date
- Decimal
- Enum
- dict (if values and keys are one of types mentioned above, or inherit one of them)
- any Iterable (if types of values are mentioned above, or inherit one of them)
If you want to add serialization of any other type or redefine the default behaviour. You should add something like this:
serialize_types = (
(SomeType, lambda x: some_expression),
(AnyOtherType, some_function)
)
To your own mixin class inherited from SerializerMixin
:
from sqlalchemy_serializer import SerializerMixin
from geoalchemy2.elements import WKBElement
from geoalchemy2.shape import to_shape
def serialize_int(value):
return value + 100
class CustomSerializerMixin(SerializerMixin):
serialize_types = (
(WKBElement, lambda x: to_shape(x).to_wkt()),
(int, serialize_int)
)
... or directly to the model:
from geoalchemy2 import Geometry
from sqlalchemy_serializer import SerializerMixin
class Point(Base, SerializerMixin):
serialize_types = (
(WKBElement, lambda x: to_shape(x).to_wkt()),
(AnyOtherType, serialize_smth)
)
__tablename__ = 'point'
id = Column(Integer, primary_key=True)
position = Column(Geometry('POINT'))
Unfortunately you can not access formats or tzinfo in that functions. I'll implement this logic later if any of users needs it.
To keep datetimes
consistent its better to store it in the database normalized to UTC.
But when you return response, sometimes (mostly in web, mobile applications can do it themselves)
you need to convert all datetimes
to user's timezone.
So you need to tell serializer what timezone to use.
There are two ways to do it:
to_dict
functionimport pytz
item.to_dict(timezone=pytz.timezone('Europe/Moscow'))
import pytz
from sqlalchemy_serializer import SerializerMixin
from some.package import get_current_user
class CustomSerializerMixin(SerializerMixin):
def get_tzinfo(self):
# you can write your own logic here,
# the example below will work if you store timezone
# in user's profile
return pytz.timezone(get_current_user()['timezone'])
If you've faced with maximum recursion depth exceeded exception, most likely the serializer have found instance of the same class somewhere among model's relationships. Especially if you use backrefs. In this case you need to tell it where to stop like below:
class User(Base, SerializerMixin):
__tablename__ = 'users'
# Exclude nested model of the same class to avoid max recursion error
serialize_rules = ('-related_models.user',)
...
related_models = relationship("RelatedModel", backref='user')
class RelatedModel(Base, SerializerMixin):
__tablename__ = 'some_table'
...
user_id = Column(Integer, ForeignKey('users.id'))
...
If for some reason you need the field user
to be presented in related_models
field.
You can change serialize_rules
to ('-related_models.user.related_models',)
To break the chain of serialisation a bit further.
Recursive models and trees
If you add controversial rules like serialize_rules = ('-prop', 'prop.id')
The serializer will include prop
in spite of -prop
rule.
If you pass rules in serialize_only
the serializer becomes NOT greedy and returns ONLY fields listed there.
So serialize_only = ('-model.id',)
will return nothing
But serialize_only = ('model', '-model.id',)
will return model
field without id
Do not forget to add comma at the end of one element tuples, it is trivial, but a lot of developers forget about it:
serialize_only = ('some_field',) # <--- Thats right!
serialize_only = ('some_field') # <--- WRONG it is actually not a tuple
To run tests and see tests coverage report just type the following command:(doker and doker-compose should be installed on you local machine)
make test
To run a particular test use
make test file=tests/some_file.py
make test file=tests/some_file.py::test_func
I will appreciate any help in improving this library, so feel free to submit issues or pull requests.