Python notes

pydantic

Pydantic

Pydantic is a data validation and parsing library for Python that uses type annotations to validate structured data and ensure correctness. It auto-generates parsing, validation, and error reporting with clean syntax.


Why use Pydantic?

  • You want automatic validation and type enforcement from untrusted input (APIs, configs, user input).
  • You want type-safe models without boilerplate code.
  • You need to define schemas for structured data (e.g. configs, API requests/responses).

When to use Pydantic?

  • When working with FastAPI or other web frameworks (it's the backbone of FastAPI models).
  • When dealing with external data sources (e.g. reading JSON, YAML, environment variables).
  • When defining configurations using BaseSettings.
  • When building data pipelines, where schema consistency is important.

⚙️ How does it work?

You define a class that inherits from pydantic.BaseModel, declare fields with types, and Pydantic does:

  • Parsing of input data (JSON, dict, etc.).
  • Type conversion (e.g., strings to dates).
  • Validation (custom rules, required fields).
  • Serialization (e.g., .dict(), .json()).
  • Error reporting with helpful messages.

Core Concepts in Pydantic

  • You define a BaseModel, annotating its fields with types.
  • Pydantic uses these types to parse incoming data, e.g., from API requests or config files.
  • It will validate fields, raise helpful errors, or coerce types as needed.
  • You can plug in custom validation if type hints aren't enough.
  • It works recursively with nested models.
  • For configs, BaseSettings integrates with env vars seamlessly.

What is Type Coercion?

Type coercion means automatically converting a value from one type to another in order to match an expected type. In Pydantic, when you define a field like this:

from pydantic import BaseModel
from datetime import datetime

class Example(BaseModel):
    number: int
    is_active: bool
    joined: datetime
Example.model_validate({
    "number": "123",           # coerced from str → int
    "is_active": "true",       # coerced from str → bool
    "joined": "2024-01-01T00:00:00"  # coerced from str → datetime
})

Pydantic tries to be helpful by accepting common alternative representations and converting them before running any validators (unless you use mode='before' in a field validator).


⚙️ When does coercion happen?

Step What Happens
1. Input received Raw input (e.g. JSON, dict)
2. Coercion Pydantic converts types (e.g. str → int)
3. Validation Type-checked, custom validators run
4. Model created A valid model is returned

Note: This order matters if you're using @field_validator(mode='before') — it sees the raw input, before coercion.


Can I disable coercion?

Yes. If you want strict type enforcement, use Strict* types:

from pydantic import BaseModel, StrictInt

class Config(BaseModel):
    count: StrictInt

Config(count=123)      # ✅ OK
Config(count="123")    # ❌ Raises error, no coercion allowed

🚧 Caveats

  • Coercion may silently succeed but introduce bugs (e.g., "False" → True)
  • Be careful when dealing with booleans, numbers from strings, or fuzzy JSON input
  • Use custom validation or Strict* types when necessary

TL;DR

  • ✅ Type coercion = converting input to match declared type
  • 🤖 Pydantic does this automatically based on your type annotations
  • 🧼 It helps reduce boilerplate and handles messy input
  • 🔒 Use Strict* or @field_validator(mode='before') to avoid or control it

Want to see a table of common coercion behaviors or a quirky edge case example?

🔍 @field_validator and @model_validator

Pydantic v2 replaces @validator from v1 with more expressive and structured decorators:

  • @field_validator → for per-field validation
  • @model_validator → for whole-model validation

These leverage new Annotated and functional syntax underneath to offer more control.


@field_validator

Used to validate or transform a single field. Can run:

  • Before type coercion (mode='before')
  • After type coercion (mode='after')
  • Wrap the whole validation (mode='wrap')
from pydantic import BaseModel, field_validator

class Product(BaseModel):
    name: str
    price: float

    @field_validator('price')
    @classmethod
    def must_be_positive(cls, v: float) -> float:
        if v <= 0:
            raise ValueError('Price must be positive')
        return v

One field can have multiple validators - just stack them:

class Product(BaseModel):
    price: float

    @field_validator('price')
    @classmethod
    def check_positive(cls, v):
      return v if v > 0 else ValueError("positive")

    @field_validator('price')
    @classmethod
    def check_reasonable(cls, v):
      return v if v < 10000 else ValueError("too high")

Note: Multiple validators on the same field run in the order defined.


@model_validator

Used to validate the entire model, useful for:

  • Cross-field validation
  • Complex object-wide logic
from pydantic import BaseModel, model_validator

class User(BaseModel):
    password: str
    confirm_password: str

    @model_validator(mode='after')
    def passwords_match(self) -> 'User':
        if self.password != self.confirm_password:
            raise ValueError('Passwords do not match')
        return self

Why are validators sometimes @classmethod, sometimes not?

It depends on the mode and context:

@classmethod

Most common with @field_validator — because the method is called without a model instance, just the raw field value.

  • You use cls because the field validator doesn’t have access to self (model instance doesn’t exist yet).
  • Even model_validator can be @classmethod when run in 'before' mode (i.e., before the model is constructed).

Instance method

Only allowed when the model instance exists:

  • @model_validator(mode='after') → full model is available, so you can use self.

🔁 Return Types: Why do validators have to return a value?

  • @field_validator must return the (possibly modified) value
  • @model_validator must return the model itself (self)
    or a modified model if needed

This is because validators act as transforms — not just checks. You can:

  • Clean/transform the data (e.g., strip strings, parse formats)
  • Raise ValueError if validation fails
  • Chain transforms by returning intermediate values
@field_validator('email')
@classmethod
def lowercase_email(cls, v: str) -> str:
    return v.lower()

🧠 Modes Summary

Decorator Purpose Runs When Uses cls or self Return Type
@field_validator Field validation before/after/wrap usually cls new field value
@model_validator Whole-model logic before/after/wrap cls or self model (self)

🛠 Practical Use Case Example

from pydantic import BaseModel, field_validator, model_validator

class SignupData(BaseModel):
    username: str
    password: str
    confirm_password: str

    @field_validator('username')
    @classmethod
    def strip_username(cls, v):
        return v.strip()

    @field_validator('password')
    @classmethod
    def password_length(cls, v):
        if len(v) < 8:
            raise ValueError("Password too short")
        return v

    @model_validator(mode='after')
    def passwords_match(self):
        if self.password != self.confirm_password:
            raise ValueError("Passwords do not match")
        return self

🧠 Tip: Use Enums to restricting values

  • Clear constraints
  • Self-documenting
  • Auto-coercion & validation
from enum import Enum
from pydantic import BaseModel

class Role(str, Enum):
    admin = 'admin'
    user = 'user'
    guest = 'guest'

class User(BaseModel):
    username: str
    role: Role

u = User(username="alice", role="admin")  # auto-casts string to Role

🧠 Tip: Use Unions for multiple possible types

from typing import Union
from pydantic import BaseModel

class Cat(BaseModel):
    species: str = "cat"
    lives: int

class Dog(BaseModel):
    species: str = "dog"
    breed: str

class PetOwner(BaseModel):
    pet: Union[Cat, Dog]

📌 Input decides which model is used based on fields or discriminator (see below for smarter matching).

Nested Models

class Address(BaseModel):
    city: str
    zip_code: str

class Company(BaseModel):
    name: str
    location: Address

class Employee(BaseModel):
    name: str
    company: Company

All nested models will:

  • Parse recursively
  • Validate deeply
  • Raise nested error messages
e = Employee.model_validate({
    'name': 'Alice',
    'company': {
        'name': 'Acme Inc',
        'location': {
            'city': 'Tokyo',
            'zip_code': 123456
        }
    }
})

🧠 Pattern: Reusable validators with complex models

You can mix all these to:

  • Build APIs that accept many input types
  • Do schema validation for configs or message protocols
  • Apply field validators to control data constraints
  • Use @model_validator to validate cross-field rules in nested models

🧩 Pydantic v2: @property, @cached_property, and @computed_field

based on the official docs and practical behavior:

💡 What is a computed field?

In Pydantic v2, @computed_field lets you include derived or lazily evaluated values (e.g., virtual fields) in:

  • .model_dump() / .dict()
  • .model_json()

These values are not stored in the model, but are computed on access.


🛠 Decorators Overview

Decorator Evaluated When Cached? Included in .model_dump()? Notes
@property On every access ❌ No ❌ No Standard Python property — not serialized
@cached_property Once, then cached ✅ Yes ❌ No Great for expensive calculations, but not serialized
@computed_field + @property On every access ❌ No ✅ Yes Serialized and schema-aware; recomputed every time
@computed_field + @cached_property Once, then cached ✅ Yes ✅ Yes ✅ Fully supported — best of both worlds
@computed_field alone (no wrapper) On every access ❌ No ✅ Yes Works, but ⚠️ loses type hints and IntelliSense

✅ Official Notes from the Docs

“Computed fields allow property and cached_property to be included when serializing models or dataclasses. This is useful for fields that are computed from other fields, or for fields that are expensive to compute and should be cached.”

“If the @computed_field decorator is applied to a bare function... it will wrap the function in property itself. Although this is more concise, you will lose IntelliSense in your IDE, and confuse static type checkers.”


🧪 Example with All Three

from pydantic import BaseModel, computed_field
from functools import cached_property

class Rectangle(BaseModel):
    width: int
    length: int

    @property
    def classic_area(self) -> int:
        return self.width * self.length

    @cached_property
    def lazy_area(self) -> int:
        print("Computing lazy_area...")
        return self.width * self.length

    @computed_field
    @property
    def virtual_area(self) -> int:
        return self.width * self.length

    @computed_field
    @cached_property
    def virtual_cached_area(self) -> int:
        print("Computing virtual_cached_area...")
        return self.width * self.length

r = Rectangle(width=3, length=2)

print(r.classic_area)             # Computed on access, not dumped
print(r.lazy_area)                # Computed once, not dumped
print(r.virtual_area)            # Computed on access, included in model_dump()
print(r.virtual_cached_area)     # Computed once, included in model_dump()

print(r.model_dump())

✅ Recommendation

If you want... Use...
Serialization + schema + dynamic value @computed_field + @property
Serialization + schema + caching @computed_field + @cached_property
Just a view / accessor (internal use) @property
Expensive value cached (not serialized) @cached_property
Shortcut syntax (but less IDE/static support) @computed_field (bare function)