How to encode custom python objects as BSON with Pymongo?

To encode custom Python objects as BSON with PyMongo, you need to write a SONManipulator. SONManipulator instances allow you to specify transformations to be applied automatically by PyMongo when storing and retrieving data.

Creating a Custom SONManipulator

Here's how to create a SONManipulator that handles custom object encoding and decoding ?

from pymongo.son_manipulator import SONManipulator

class Transform(SONManipulator):
    def transform_incoming(self, son, collection):
        for (key, value) in son.items():
            if isinstance(value, Custom):
                son[key] = encode_custom(value)
            elif isinstance(value, dict): # Make sure we recurse into sub-docs
                son[key] = self.transform_incoming(value, collection)
        return son
    
    def transform_outgoing(self, son, collection):
        for (key, value) in son.items():
            if isinstance(value, dict):
                if "_type" in value and value["_type"] == "custom":
                    son[key] = decode_custom(value)
                else: # Again, make sure to recurse into sub-docs
                    son[key] = self.transform_outgoing(value, collection)
        return son

Registering the SONManipulator

Add the manipulator to your PyMongo database object ?

db.add_son_manipulator(Transform())

Complete Example with Custom Class

Here's a complete example showing how to encode a custom Person object ?

import pymongo
from pymongo.son_manipulator import SONManipulator

class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age

def encode_person(person):
    return {
        "_type": "person",
        "name": person.name,
        "age": person.age
    }

def decode_person(data):
    return Person(data["name"], data["age"])

class PersonManipulator(SONManipulator):
    def transform_incoming(self, son, collection):
        for (key, value) in son.items():
            if isinstance(value, Person):
                son[key] = encode_person(value)
            elif isinstance(value, dict):
                son[key] = self.transform_incoming(value, collection)
        return son
    
    def transform_outgoing(self, son, collection):
        for (key, value) in son.items():
            if isinstance(value, dict):
                if "_type" in value and value["_type"] == "person":
                    son[key] = decode_person(value)
                else:
                    son[key] = self.transform_outgoing(value, collection)
        return son

# Usage
client = pymongo.MongoClient()
db = client.test_database
db.add_son_manipulator(PersonManipulator())

# Now you can store Person objects directly
person = Person("Alice", 30)
db.users.insert_one({"user": person})

Key Points

• transform_incoming converts custom objects to BSON−compatible format before storage
• transform_outgoing converts stored data back to custom objects when retrieving
• Use a _type field to identify custom objects during decoding
• Recursively process nested dictionaries to handle complex document structures

Note: You don't have to add the _type field if you want to silently cast objects like NumPy arrays to Python arrays without explicit type identification.

Conclusion

SONManipulators provide automatic encoding/decoding of custom Python objects in PyMongo. Implement transform_incoming and transform_outgoing methods to handle your custom object serialization needs.