A plugin for Laravel Filament that manages data source connections.
Repo phpinnacle/ferry, read from GitHub 37 min ago. phpinnacle has not joined Take My Repo.
phpinnacle/ferry is a public repo on GitHub by phpinnacle. A plugin for Laravel Filament that manages data source connections. It declares the MIT license on GitHub. It is written mainly in PHP, has 0 stars and last changed on 2026-11-18. Nobody has reviewed it on Take My Repo yet, which says nothing about its quality. Read from GitHub on 2026-11-22.
In the maintainer's words. Shown from GitHub; Take My Repo does not keep a copy.
Ferry manages upstream databases, source structure snapshots, replica mappings, and CDC origins in Filament. The Rosetta adapter reads 1C structures for replication; the Eloquent adapter captures application model tables for model events. The built-in Kafka origin integrates Debezium and JDBC connectors through Kafka Connect. Its forms use reusable FieldMapping and FieldBinding fields from Eidos.
Upstream model with encrypted adapter configuration and ordered source record transforms.UpstreamFactory service that tests current (possibly unsaved) form values, always closes the temporary connection, and reports a sanitized result.Adapter, with a Rosetta adapter for 1C and ready JSON Schemas stored as StructureItem records.UpstreamResource with create/edit/list pages, a structure status badge and configuration forms supplied by the selected adapter. Rosetta provides "Test Upstream" and "Fetch structure" actions.phpinnacle/eidos, with Ferry-owned transformation settings and shared JSON Schema compatibility.Replica model storing the source object, a saved input JSON Schema snapshot, native mapping rows, ordered record transformations and the generated destination table name.ReplicaResource that selects an active upstream with a ready structure and an object from the published snapshot, maps source fields to column names and keeps the physical destination table in sync with the record.phpinnacle/franz: one Debezium source connector per upstream, one JDBC sink connector per replica, with status and applied configuration stored in Connector models.UpstreamPolicy, ReplicaPolicy), custom database connection and optional tenancy.composer require phpinnacle/ferry
php artisan vendor:publish --tag="phpinnacle-ferry-migrations"
php artisan migrate
The package publishes a single create_ferry_tables migration for fresh installations. It creates connectors, upstreams, structures, structure_items, and replicas on the configured Ferry connection. Rollback drops these service tables and preserves replica destination tables. The required upstreams.origin column stores the selected origin configuration key without a database default.
Register FerryPlugin::make() in the target Filament panel. Publish phpinnacle-ferry-config when using a non-default database connection, tenant model or timeout. All Ferry jobs use the application's default queue connection and the queue selected by phpinnacle-ferry.queue. The default null uses that connection's default queue; run php artisan queue:work --timeout=120. For a dedicated queue, set queue to ferry and run php artisan queue:work --queue=ferry --timeout=120.
Connector management talks to Kafka Connect over PSR-18, so the application binds the transport it prefers. Kafka dependencies are resolved when a connector operation begins; constructing observers or opening a replica form does not initialize them:
use GuzzleHttp\Client;
use GuzzleHttp\Psr7\HttpFactory;
use Psr\Http\Client\ClientInterface;
use Psr\Http\Message\RequestFactoryInterface;
use Psr\Http\Message\StreamFactoryInterface;
$this->app->bind(ClientInterface::class, Client::class);
$this->app->bind(RequestFactoryInterface::class, HttpFactory::class);
$this->app->bind(StreamFactoryInterface::class, HttpFactory::class);
Upstreams require an explicit origin and adapter, both read-only after creation. Adapter-specific settings live in config, cast as encrypted:array in a text column and hidden from model serialization. Adapter::form() supplies the configuration fields; changing the adapter during creation resets those settings.
For Rosetta, config contains Laravel database credentials, including driver. Driver::schema() supplies the connection fields; changing the driver during creation resets them and applies its default port. The driver remains read-only when editing. PostgreSQL offers schema and an optional sslmode selector backed by Enums\SSLMode: disable, allow, prefer, require, verify-ca, and verify-full. The selected mode remains a native string; leaving it blank keeps the driver's default behavior. Editing fills the password field with the saved password, masked until revealed. Clearing the field clears the saved password. UpstreamFactory::make($upstream->config) creates a temporary Laravel connection; its caller must disconnect it. Eloquent uses a registered application connection instead; see Eloquent change events.
Enable the optional Schema Registry pages in a panel:
use PHPinnacle\Ferry\FerryPlugin;
FerryPlugin::make()->schemaRegistry();
Set phpinnacle-ferry.schema_registry.base_uri through FERRY_SCHEMA_REGISTRY_URL
to the Registry REST API base URL. Rosetta also uses this URL to select Avro
serialization for its source and sink connectors; without it, Rosetta uses
schemaful JSON. Eloquent always uses schemaless JSON. The viewer's HTTP connection
is separate from Kafka Connect. Configure
phpinnacle-ferry.schema_registry.headers for authentication; requests use the
existing phpinnacle-ferry.timeout setting. Enabling the viewer does not change
connector settings or register schemas.
The application must authorize the view_schema_registry gate. The viewer shows
the configured registry across subjects, without applying Ferry tenant filters;
grant this ability only to users allowed to inspect that registry.
Integrations → Schema Registry lists each active subject's latest schema,
format, reference count, and version. The table reads GET /schemas with
latestOnly=true, searches through subjectPrefix, and paginates through
offset and limit on the server. Pagination offers previous and next pages;
the API does not return a total count. Opening a subject displays its latest schema;
the page header offers a version selector, newest first, and Refresh.
The Details section contains the schema ID, version, format, and effective
compatibility badge, including the Registry's global fallback. Separate tabs show
the structure, original source, and references; References appears only when
the selected version has references.
The table and infolists use REST API data directly, without Eloquent models or
database copies. Refresh fetches current Registry state. API failures are shown
separately from empty results.
Show deleted includes soft-deleted schemas alongside active schemas and resets
pagination while preserving the subject prefix. Hide deleted returns to the
active list. The mode is kept in the deleted URL parameter and carried into
subject versions, schema source, Avro reference resolution, and the return link.
Responses that include a true deleted flag are marked Deleted; the version
page shows this badge on the right of the Details heading. Permanently
deleted schemas cannot be retrieved.
Deletion requires separate gates: delete_schema_registry for soft deletion and
force_delete_schema_registry for permanent deletion. Both also require page
access through view_schema_registry. The table offers subject deletion, which
removes all active versions and the subject compatibility configuration. The
subject page offers Delete for its selected numeric version; after success it
returns to the list, preserving deleted mode. Each row and version page offers soft
deletion for an active schema or permanent deletion for a soft-deleted schema,
according to the API's deletion status. All deletions require confirmation
identifying the target and mode.
Soft deletion retains schema IDs for reading existing messages. Permanent deletion
is available in Show deleted mode and sends a separate DELETE request with
permanent=true. It can make existing messages unreadable and cannot be undone.
The Registry enforces prior soft deletion and reference constraints; the UI never
cascades to dependencies or automatically soft-deletes before permanent deletion.
Registry failures produce an error notification without a success transition.
Deleting the last version also removes the subject compatibility configuration.
See Confluent's schema deletion guidelines.
The subject page also offers Restore for a soft-deleted selected schema,
requiring the separate restore_schema_registry gate and page access. This action
is not available in the table or bulk-action menu. After confirmation, it retrieves
the original definition and re-registers its schema, format, references, metadata,
and rules through POST /subjects/{subject}/versions. It then looks up the active
registration and displays the Registry-assigned version on the same page, updating
the version selector immediately. The version number may change. The Registry
applies current compatibility and reference constraints; dependencies, deleted
subject configuration, and Registry modes are not changed. Permanently deleted
definitions cannot be restored here.
If registration succeeds but the follow-up lookup fails, refresh before retrying.
The bulk-action menu offers one Delete selected subjects action for the current page. Active subjects are soft-deleted; already soft-deleted subjects are deleted permanently. Confirmation shows the number of subjects for each operation and warns that permanent deletion cannot be undone. The selection and deletion modes are fixed when confirmation opens; subjects whose selection or state changes before submission are skipped. Each subject requires the same gate as its individual deletion action; unauthorized subjects are skipped. Requests run per subject without a cross-subject transaction. Processing continues after a Registry rejection, refreshes the table, clears the selection, and reports the number deleted when some subjects fail or are skipped.
Avro structure previews use Eidos's inline JsonSchemaEntry with a JSON Schema
presentation built by Services\AvroSchemaConverter. It presents records,
nullable unions, named types, enums, arrays, maps, fixed values, and logical types.
Avro defaults and nullability remain independent: every record field is present
in required, including nullable fields and fields with defaults. Logical types
retain their underlying representation and show their logical name as a format.
Kafka Connect and Debezium annotations are interpreted within records, arrays,
maps, and nullable unions:
| Annotation | JSON Schema presentation |
|---|---|
connect.type: int8 / int16 on Avro int |
Integer bounds of -128–127 / -32768–32767 |
io.debezium.data.Uuid |
String with uuid format |
io.debezium.data.Enum |
String with enum values from connect.parameters.allowed, preserving spaces, empty values, and order |
io.debezium.data.Json / Xml |
String with json / xml format and application/json / application/xml content media type |
io.debezium.time.Date |
Integer days since the epoch with date format |
io.debezium.time.Time / MicroTime / NanoTime |
Integer with time-millis / time-micros / time-nanos format |
io.debezium.time.Timestamp / MicroTimestamp / NanoTimestamp |
Integer with timestamp-millis / timestamp-micros / timestamp-nanos format |
io.debezium.time.ZonedTimestamp / IsoTimestamp |
String with date-time format |
io.debezium.time.ZonedTime / IsoTime |
String with time format |
io.debezium.time.IsoDate |
String with iso-date format, preserving Debezium's UTC suffix |
io.debezium.time.Year / MicroDuration / Interval |
Original representation with year / duration-micros / duration format |
Debezium names above come from connect.name; explicit Avro logicalType takes
precedence over the presentation format. Unknown annotations retain their Avro
representation. Original metadata remains in x-avro, including
connect.parameters.__debezium.source.column.type, .length, and .scale.
These describe the source SQL column and do not impose constraints on values
that might have been changed by source transforms. No record values are converted.
See the Debezium data type mappings.
This projection is for inspection, not Avro validation or compatibility checks. Referenced Avro schemas are resolved by the Registry API for the preview; the Source and References tabs retain the registered definition and references.
JSON Schema definitions use Eidos directly. Recursive schemas and unsupported tree formats, including Protobuf, show an explicit notice and remain available in the Source tab. Source is formatted for reading; copying preserves the registered text. Arbitrary schema registration and compatibility changes are not exposed.
Every upstream owns one Models\Structure, available through $upstream->structure. It stores the preparation status, revision, run id, progress and adapter context on the Ferry database connection. Upstream.adapter stores the required adapter key: 1C Rosetta (rosetta) or Eloquent (eloquent). Creation starts structure preparation; changing adapter settings starts a new run after commit. Eloquent currently publishes an empty structure and provides table capture without replica mapping support. Structure jobs live under Jobs\Structure and carry the upstream id, run id and chunk offset; credentials never travel through the queue.
The preparation action in both the table and edit form requires the upstream's update policy permission (update_upstream). Rosetta reads the source in bounded chunks and writes objects into a draft revision. The published snapshot is replaced only once the whole draft is complete, so a failed refresh never damages the previous one. A source without supported objects is a valid empty snapshot.
use PHPinnacle\Ferry\Models\Upstream;
$upstream = Upstream::available()->firstOrFail();
foreach ($upstream->structure->publishedItems()->get() as $object) {
$object->schema; // PHPinnacle\Jason\Schema
$object->primary_key; // ordered physical primary-key columns
$object->schema->annotation('x-ferry-primary'); // ordered physical primary-key columns
$object->values; // list<array{id: string, label: string, title: string}>
$object->children; // child StructureItem models
}
Each StructureItem stores a ready object JSON Schema in schema. Its properties keys are physical source column names, used directly in mappings and connector configuration. Types, bounds, titles and descriptions are prepared by the adapter. field($source) returns the logical Schema with title, description, and x-ferry-trans removed, preserving defaults and boolean definitions without changing the stored schema. The ordered primary_key array describes the physical primary key and supplies the default key for dynamic destinations. Static replica keys come from their destination mappings. Optional code, kind, reference_id and values preserve source metadata as native scalar values and arrays. See JSON Schema extensions for the complete annotation contract.
The Rosetta adapter reads primary keys for root and child tables in batches of up to 500 tables per catalog query and stores their ordered column names in both primary_key and the object-level x-ferry-primary annotation, and marks those properties required in JSON Schema. A table without a declared primary key gets []; unique indexes are not substituted. For example, a catalog can declare ['_idrref'], while a tabular section can have a composite key. Dynamic destinations require every effective source key component to be explicitly mapped, or assembled into a UUID key through the record editor. Static destinations select their key through mappings to the destination's ordered getPrimaryKey() fields; they need neither a source PRIMARY KEY nor mappings for its components. Source connectors capture the selected physical key dependencies and use them in message.key.columns; generated fields are never sent as physical columns.
Built-in field schemas store a full translation key in x-ferry-trans. The interface resolves it in the current locale with priority over title. Recorder columns use Recorder · Reference type for _recordertref and Recorder · Reference for _recorderrref. Other fields use their stored title. The adapter sets priority for source ordering; the interface sorts equal priorities by title. The Rosetta adapter gives catalogs priority over other objects and puts obsolete titles starting with ( last.
The Rosetta adapter converts properties into independently mappable physical source columns before persistence. Single references use _fld…rref; system fields already contain their physical names. Composite properties expand into separate source cards for the discriminator (_type), any boolean (_l), number (_n), date/time (_t), string (_s), and reference identifier (_rrref) columns. A reference type code (_rtref) is included when the composite permits multiple reference targets. A composite with one reference target uses _rrref without _rtref. Nested type definitions are expanded before resolving columns.
Each card has its own type, transforms, and optional destination. Bind any subset of non-key fields: choosing an identifier does not automatically select its discriminator or reference type code. Dynamic destinations require every effective source key component to be bound. Static destinations require source bindings for every destination key component. Debezium exposes the one-byte _type discriminator as a hexadecimal string of 2 characters. The source converters expose _rtref, _recordertref and _keyfield as integers, and reference identifiers as UUID strings. Rosetta prepares these converted types in JSON Schema so mapping checks and destination columns match the captured values. Persisted source identifiers are physical column names, such as _fld17_rrref. A renamed source column requires updating its bindings. StructureItem::mappableFields() returns JSON Schema definitions keyed by physical column name, with localized titles and preserved descriptions.
Structure::publishedItems() returns the root objects of the published revision; publishedItem($source) finds one of them by its external id and returns null when absent. Use $upstream->structure->items() to reach every stored revision. $upstream->is_active states whether the upstream may be used, while $upstream->structure->status reports whether its local structure is ready.
When a run fails for good, its structure switches to StructureStatus::Failed, keeps its previous snapshot and reports the original exception through report(). Connection tests return a notification without persisting their result. Structure progress uses processed and estimated; progress updates advance the structure’s own updated_at, which detects interrupted runs independently of upstream edits. Structure::canPrepare($staleAfter) and prepare($staleAfter) accept the staleness threshold in seconds; the panel action reads it from phpinnacle-ferry.structure.stale_after. Saving a new non-null run ID triggers StructureObserver to dispatch preparation after the transaction commits.
Contracts\Adapter owns configuration forms, structure reading and capture planning. Adapters declare getKey(): string, getLabel(): string, form(): array, capture(Upstream): CapturePlan, prepare() and read(). Services\StructureAdapterRegistry registers adapters by getKey(), exposes their labels through options(), and resolves the saved key through getOrFail(). The package registers Services\RosettaAdapter under rosetta and Services\EloquentAdapter under eloquent; there is no automatic fallback. Register additional implementations from an application service provider:
use PHPinnacle\Ferry\Services\StructureAdapterRegistry;
$this->app->make(StructureAdapterRegistry::class)->register(
$this->app->make(CustomStructureAdapter::class),
);
prepare(Upstream) returns a Data\StructurePlan with an estimated root count and adapter-owned serializable context. read(Upstream, StructurePlan, $offset, $limit) returns unsaved StructureItem roots with their children relation loaded. Jobs resolve the selected adapter for each preparation or chunk. Adapters close their source connections and do not save models, publish revisions or dispatch jobs.
Jobs persist the returned objects through Structure::importItems(). The structure owns metadata identities, draft publication and stale-run checks. StructureObserver dispatches preparation after commit when a new run ID is saved; saving progress or a plan does not enqueue another preparation job. Rosetta’s storage and type maps are encoded only by RosettaAdapter; other adapters can use different context. Rosetta fields and composite expansion stay inside the structure adapter under Services\Rosetta. Models, mappings, validation and Kafka configuration consume ready schemas. Other adapters return the same StructureItem contract without Rosetta types.
StructureItem uses an upstream_id foreign key. The one-to-one Structure uses the same key as its primary key and exposes its items through items().
The creation form combines destination type and destination in one searchable Destination select: Dynamic first, followed by all registered static destinations. The helper field _destination is not saved; hidden type and destination attributes are derived from the validated selection on submission. The creation form collects general settings only; mapping and transformations are configured on the edit page. Upstream, source and destination use disabled selects when editing; destination type is not an edit field. Destination displays the registered static label or the saved dynamic table name. Changing selections during creation does not initialize mapping or transformation state. The editor reads the source schema and destination from the saved replica and submits only name, mapping and record transformations.
A replica selects an active upstream with a ready structure and one object from its published snapshot, then maps source fields by their physical column names to destination columns. For dynamic destinations, column names must be unique, start with a lowercase letter, and contain only lowercase letters, digits and underscores. Every effective primary-key column must be explicitly bound to a destination column before a mapping can be saved; constants do not satisfy a key binding. Key fields display required markers on the left of the editor. A source without a primary key needs a generated UUID key to configure a dynamic destination; static destinations derive their key from the mapping. Creating a pending replica creates no destination table. The first complete mapping save creates {prefix}{code} in the Ferry connection, with non-null key columns and a primary key in the effective replica key order. Other mapped columns are nullable and typed from the source, their transformations or static values.
The record transformation editor is a Common TabbedBuilder above the mapping. Its catalog contains every registered transform with TransformScope::Record. Built-in tabs show their type and selected source fields (UUID: tenant_id, row_id, Slug: code) with a type icon. Both forms place Field name first. Add, edit, remove or reorder native tabs; their saved order is the record pipeline order. The record and mapping editors stay in place while the source schema follows current record settings. Incomplete settings remain editable and do not produce a generated field until their output can be determined. Saving validates every tab, including inactive tabs, and rejects invalid dependencies or order.
The Generate UUID key tab uses the Eidos JSON Schema picker to select one or more fields from the incoming record, including generated fields from preceding transforms, and an editable output field name, defaulting to id. It is available without x-ferry-primary; when present, the incoming key supplies the default selection. Selections are stored as JSONPath strings in selection order, and the connector receives their field names in that order. The effective schema adds a UUID string of length 36 and replaces its x-ferry-primary with the output field; bind that field to the destination key. The physical input snapshot remains unchanged. The shared source captures the selected fields' physical dependencies in message.key.columns and never captures generated fields. Choose fields whose combined values uniquely identify each row; Ferry does not check data uniqueness.
The sink runs record transformations against Value and repeats the selected key dependencies against Key, including org.phpinnacle.toblerone.InsertUuid$Key, before selecting and renaming key fields. JDBC reads the UUID from record_key for both upserts and tombstone deletes; no value placeholder is inserted. Install this SMT on the Kafka Connect worker. Its existing default UUID namespace is used.
Generated UUID and slug fields have a title derived from their output name (slug → Slug, record_id → Record Id). Both mapping editors show this title beside the technical name, using the same heading style as titled source fields.
The Generate slug tab starts with the output field name, followed by the Eidos JSON Schema picker with single selection for a string input and optional additional separator characters. It appends a slug setting to replica.transforms, adds a normal mapping row and places the generated string immediately after its source field in the effective schema and both mapping editors. The SMT preserves null values and source optionality; it does not impose uniqueness or a length limit and may produce an empty string. A generated field can have its own subsequent field transforms. Generated output names use lazy state binding: leaving the field updates the mapping sources without saving the replica. Deleting a generated output updates the mapping sources; unrelated mappings and constants are retained. Renamed outputs need to be bound again. Inputs may refer to earlier generated fields, including UUIDs. Field names cannot replace existing properties.
Replica::schema stores the physical input snapshot. Services\ReplicaSchemaResolver::transformedSchema($replica) applies replica.transforms to that snapshot before field transforms or destination compatibility checks. ReplicaSchemaResolver::previewRecordSchema($schema, $settings) takes typed TransformSetting objects and returns the last valid schema before an incomplete or incompatible setting; the form and dependent tabs share this preview. Record transforms declare sourceFields() so the shared source captures their physical dependencies even when those fields have no destination mapping. Generated columns are never requested from Debezium.
ReplicaSchemaResolver receives TransformRegistry through constructor injection and accepts replicas as method arguments. primaryKeySources($replica) resolves the effective source key for dynamic destinations. Replica::primaryKeyMap($destination->getPrimaryKey()) returns the source-to-destination key map in destination key order for either destination type. keyPlan($replica, $sources) resolves selected fields to their physical dependencies and ordered record transforms. Replica::columnMap() and constantColumns() read the saved mapping directly. Models own state and business rules that do not require services; schema resolution and SQL table planning belong to services.
String columns use the effective output schema calculated from the saved input schema and transforms: a positive maxLength produces varchar(n), while an absent or zero limit produces varchar without a length restriction. Untransformed source strings use their saved input schema's length; untransformed static strings create unbounded columns. Transformations such as Mask that discard the source length therefore produce an unbounded column. Changes to string length use ALTER COLUMN and preserve stored values, including when the column is renamed. PostgreSQL rejects narrowing a column if existing values exceed the new limit, and the replica save rolls back.
Editing the mapping applies the difference to the table: new mappings add nullable columns, renames rename the column and removed mappings drop it. Changing the effective key updates the table's PRIMARY KEY and its column order even when the mapped columns stay unchanged. New key columns are non-null; retained former key columns become nullable. Existing rows must satisfy the new key's uniqueness and non-null constraints, otherwise the save rolls back. Adding a generated UUID key does not backfill existing rows. Deleting the replica requests cleanup after a data-loss confirmation; the worker removes its sink before dropping the dynamic table. Structure refreshes do not rewrite replica snapshots or pause replicas.
Removing a mapping or changing its SQL column type applies immediately when the form is saved, without an additional confirmation. A type change drops and recreates the column; both operations discard the affected column data.
ReplicaObserver delegates dynamic table creation and schema changes to ReplicaTableManager whenever a replica is saved, regardless of the caller. Saving a configured mapping also builds the source and sink configurations through Origin::make(). The service saves both connectors and their owner associations in the same transaction as the replica. Connector observers dispatch jobs after that transaction commits; rollback discards the state and table changes without queueing connector operations. Deletion retains the owner until the worker removes its connector and drops the dynamic table.
Replica.schema stores the complete input JSON Schema copied from the selected source when the model is created, including its metadata and translations. The editor uses this saved snapshot directly; it neither submits the schema nor reloads it from the current upstream structure. Effective output schemas are calculated through the shared transformation definitions for validation, column creation and column changes.
Replica.mapping uses the same native list as the Eidos fields:
[
['kind' => 'source', 'source' => 'record_id', 'target' => 'id', 'options' => ['transforms' => []]],
['kind' => 'static', 'target' => 'active', 'static' => true, 'options' => []],
]
A nullable target leaves a source unbound. Static rows with target: null are editor drafts and do not create columns or connector transforms. options preserves arbitrary source settings as a nested array; Ferry reads its ordered transforms entries. Rows contain no input or output schema copies. Data\FieldMapping::fromArray() converts these rows into typed mappings. ReplicaSchema::assertValid($mappings, $destination, $sourcePrimaryKey, $sourceSchema) checks effective output types, required destination fields and primary-key mappings at the form boundary. Native Filament forms validate transform parameters and dynamic column names.
ReplicaTableManager::mappingChanges($replica) plans additions, renames, removals, SQL type replacements, and string length alterations from the original and current mappings, input schemas and transforms. Table updates use the same plan and preserve stored values on renames.
The status field is the only replica lifecycle state. Administrator-requested pauses use Pause. Saving a paused replica preserves that status; resume it with Start once its mapping is valid. A replica remains Pending until its connector is created in the origin. The initial empty mapping is a draft and does not create connectors; saving its configured mapping queues deployment. Saving an active or paused replica updates its configuration without changing its requested pause state.
A replica writes either into a generated {prefix}{code} table (dynamic) or into a table the application already owns (static). DynamicDestination and StaticDestination independently implement the Destination interface, which exposes presentation metadata, getTable(), getConnectionName(), getPrimaryKey(), and getFixedValues(). Primary keys are always non-empty ordered lists, including single-column keys such as ['id']; getPrimaryKey() returns that list. Origins consume this destination data and own the external replication configuration. Dynamic destinations derive their ordered primary key from the replica's explicit source mappings and use its generated table name; static destinations describe the application's target and its mapping fields. Register static destinations on the plugin:
use PHPinnacle\Ferry\Destinations\StaticDestination;
use PHPinnacle\Ferry\FerryPlugin;
use PHPinnacle\Jason\SchemaBuilder;
FerryPlugin::make()->destinations(new StaticDestination(
key: 'customers',
label: 'Customers',
table: 'customers',
primaryKey: ['id'],
schema: SchemaBuilder::object([
'id' => SchemaBuilder::string()
->format('uuid')
->title('ID')
->build(),
'name' => SchemaBuilder::string()->title('Name')->build(),
'manager_id' => SchemaBuilder::string()
->format('uuid')
->title('Manager')
->build(),
])
->required(['id', 'name'])
->build(),
connection: 'sales',
fixedValues: ['type' => 'customer'],
));
The readonly key identifies the destination in the persisted replica. schema accepts a PHPinnacle\Jason\Schema or a closure returning one, exposed through getSchema(). Use a flat object schema with string, integer, number, or boolean fields (optionally nullable): properties declares columns, title supplies labels, and required identifies mandatory mappings.
String and integer destination properties can declare x-lookup to choose static
values by record label through Eidos's built-in mapping editor:
'category_id' => [
'type' => ['integer', 'null'],
'x-lookup' => [
'model' => App\Models\Category::class,
'label' => 'name',
'search' => ['name', 'code'],
'key' => 'id',
],
],
Only model is required: label defaults to name, search to [label],
and key to the model's primary key. The editor saves the selected key and
preloads up to 20 records, ordered by label and key. It resolves the saved key's
label when reopened, respecting the model's connection and global
scopes. Integer and string keys retain their schema types. Clearing a nullable
lookup omits the static mapping, following the existing empty-value rule.
Destination primary-key fields still require source connections and never offer
static editors, including when annotated with x-lookup. Saved values remain
subject to destination schema validation.
table, connection, primaryKey and fixedValues configure the target independently of its schema. A null connection uses the Ferry connection. Fixed values use an array<string, mixed> map and are written into every row through InsertField transforms. Boolean, integer and floating-point constants receive a matching Cast after insertion; JSON Schema default does not insert values.
Destination extends Filament’s HasLabel, HasIcon, and HasColor contracts. Static destinations accept optional icon and color constructor arguments, and label may be a closure. Use label and schema closures for translations and catalogs that must reflect the current request. Dynamic destinations use their table name as the label and have no icon or color by default.
The replica list resolves destinations through DestinationFactory::resolve() and displays them as badges using their labels, icons, and colors. Dynamic destinations display their table name. The list has no separate type column. After Upstream, its columns are Destination, Connector, and Status. Both upstream and replica lists label the connector status column Connector. Failed connector badges show the saved connector error in a native plain-text tooltip when an error is available. Upstream and Replica edit forms show the same error in a native danger callout above the form; it is hidden when the connector is not Failed or no error is saved. The replica list and edit-page header show the same lifecycle controls: Pause when the replica is Active and its connector is Running, otherwise Start. The edit-page actions use the saved replica without saving form drafts. Start retries a failed connector without rebuilding its configuration; a paused replica is activated first so the retry resumes it. There is no separate Restart action. Both lists show the Refresh statuses header action before their Create action. It checks every saved source and sink connector, independent of filters and pagination. It requires permission to view both the replica and upstream lists. Per-record status checks are absent from both lists. External failures are reported together while checks continue for other connectors; owners without a saved connector are left unchanged. Replicas can be filtered by status, upstream, and destination, separately or together. Destination options use registered labels for static destinations and table names for dynamic destinations.
JSON Schema is the field contract throughout Ferry. References use type: string, format: uuid, minLength: 36 and maxLength: 36. Dynamic SQL tables store them as varchar(36), and transforms and mapping compatibility treat them like other UUID strings. Temporal values use the date, time or date-time format and create SQL date, time or datetime columns respectively.
Replica.type explicitly selects DestinationType::Static or DestinationType::Dynamic (the default). Replica.destination stores the registered key for static replicas or the generated table name for dynamic ones. DestinationFactory::get($key) returns the registered destination or null. Use getOrFail($key) when a static destination is required. DestinationFactory::resolve($replica) returns the registered static destination or a DynamicDestination for its generated table. staticFor($replica) returns the registered static destination, or null for a dynamic or unavailable destination. Replica initializes its dynamic destination table name from the configured prefix and code in its creating lifecycle event. The saved name stays unchanged when the prefix or code changes. DynamicDestination receives the replica and ReplicaSchemaResolver, reading the initialized table name and resolving the effective mapped key; resolve dynamic destinations after the replica has been initialized. Static records retain their registered destination key.
Saving a static replica accepts only declared destination fields compatible with the output JSON Schema. The mapping editor and saving use directional compatibility for types, nullable values, enum subsets, formats and string lengths. An integer can feed a number; a formatted string can feed plain text. UUID strings and references with the same constraints are compatible. When the output enum is a subset of the destination enum, formats do not restrict the connection; type, nullability and length checks still apply. Native static forms validate value formats; saved constants supply their native type, exact string length and enum value for compatibility checks. These schema checks do not validate streamed records. Every required field and every destination key field must be mapped, and no destination may be mapped twice. A destination key field accepts any schema-compatible source field, including a generated field, but never a constant. Physical source key fields can map to ordinary destination columns or stay unbound. For example, _idrref → Customers.id makes _idrref the replica and Debezium message key regardless of StructureItem.primary_key. The user is responsible for uniqueness; Ferry does not query data or infer unique indexes. Static sinks use pk.mode: record_key, select and rename the mapped key, and list destination key fields in their declared order in pk.fields. Declare every key column in the destination schema. Dynamic destinations retain their explicitly mapped effective source key. Ferry never creates, alters or drops a static table; its sink upserts rows and propagates source deletions by the mapped key. An unavailable registered destination blocks saving and activation until it is registered again.
Replicas are created from the list page in a modal with the replica icon and a description of the next step. Name, Code, Upstream, Source, and Type appear directly in the modal without a section heading or frame. Type selects a dynamic table or a static destination; static replicas select their registered Destination separately. Creation saves an empty mapping in Pending status and redirects immediately to the edit page. Code, upstream, object, and destination are immutable in that form; the destination remains a disabled select. Only the name, record transforms, and mapping can be edited. Required destination mappings are validated when saving the edit form, so static replicas can be created before configuring their fields. The resource has no separate create page. Forms load model attributes directly and return validated attributes from getState() for saving. The model casts accept native schema and transform arrays and expose typed values to domain code; no page-level data conversion is required.
The creation form has a Type selector and a separate Destination selector shown for static replicas. A dynamic replica lists every mappable 1C field with its title and physical column name and binds it to a column name with FieldBinding; a static one connects 1C fields to the fields declared by the destination with FieldMapping, which shows the logical types and required marks and refuses incompatible connections. Static destination mappings use Eidos's built-in static forms for non-key destination fields: unconnected cards show native fields derived from their JSON Schema, including string limits, enums, numeric inputs and Boolean toggles. Values retain their native types without a Cast. Static editors omit null values during native dehydration; Boolean false and numeric zero are preserved. Dynamic destinations keep the Add static value workflow for source cards and transforms. Replica constants remain subject to the replica schema's scalar value and destination type requirements.
The Upstream, Source, and Destination selectors use Filament JavaScript selects with unselectable placeholders. Upstream and Destination preload all options without a display limit. Source preloads at most 20 published sources; server-side search matches titles, labels, and physical names and returns at most 20 results. The selected label is loaded separately, so sources outside the initial list remain selectable. Source options are ordered in the database by priority and case-insensitive display title, followed by external ID; numeric parts use database alphabetical order. New replicas default to Dynamic, the first active upstream with a ready structure in name order, and its first object in the displayed order. Changing the upstream selects its first available object and clears the mapping. Empty option lists leave the corresponding selection unset. Both replica mapping editors use dense() for reduced spacing and compact() to show descriptions in title tooltips. Dense mode reserves one row for transforms so adding the first transform does not increase the card height; wrapped transforms can expand the card.
The Rosetta adapter prepares source constraints once when importing the structure. Numeric source schemas additionally retain exact storage dimensions in x-ferry-decimal: [precision, scale]; an unknown dimension is null, and the annotation is omitted when both are unknown. Dynamic decimal columns use these values, defaulting to 15 and 2 respectively for missing or null dimensions. Strings with a positive length expose maxLength; fixed strings also expose the same minLength. Equal lower and upper limits are displayed as one value, such as 9, instead of 9–9. Zero or unknown length is unbounded. Non-negative numbers expose minimum: 0. Known positive scale becomes multipleOf: 10^-scale. When precision and scale are both known, the numeric range uses exclusiveMaximum: 10^(precision-scale) and, for signed numbers, its negative as exclusiveMinimum. Integer types and the decimal step exclude the endpoints and express the permitted values without rounding a decimal maximum full of nines. The editor shows these as schema constraints rather than generated descriptions. Unknown dimensions do not produce inferred bounds. Rosetta property comments populate description for their scalar field or every physical column of a composite field. Missing comments leave the description absent. Compact cards show comments in title tooltips; type metadata remains in schema constraints and is never appended to the comment.
Date/time sources require an explicit Cast before connecting to a plain string destination in the static replica form. The reusable FieldMapping component keeps Eidos's more general directional format compatibility.
Ferry uses the following schema annotations. Structure adapters write them
into StructureItem.schema, whose property keys are physical column names.
Jason preserves these annotations, and Eidos leaves their meaning to Ferry.
| Annotation | Value | Scope and behavior |
|---|---|---|
x-ferry-primary |
Ordered list of strings | Object-level key columns. In StructureItem.schema, Rosetta writes the physical primary index and saves the same order in StructureItem.primary_key. In Replica.schema, this identifies the input key. Record transforms may replace it in Services\ReplicaSchemaResolver::transformedSchema($replica), including with a generated UUID field. Key properties are listed in required. Dynamic destinations use the effective record key. Static destinations use their destination mappings. Source capture resolves selected generated key fields to physical dependencies. |
x-ferry-trans |
String | Full Laravel translation key for a source property label. Resolved in the current interface locale and takes priority over title. Used for built-in fields. |
x-ferry-converter |
String | Source converter key from phpinnacle-ferry.origins.<name>.source.converters. Rosetta infers uuid for physical names ending in rref, and integer for _keyfield, _recordertref and names ending in _rtref, including expanded composite columns and tabular sections. It prepares UUID reference string schemas or integer schemas respectively. Other columns omit it, including _type discriminators and ordinary UUID fields such as _predefinedid. The Kafka source groups captured columns by this annotation to generate <key>.columns; it does not infer converters from column names. |
x-ferry-decimal |
Two-element array [precision, scale] |
Retains SQL numeric storage dimensions alongside standard JSON Schema constraints. Precision is the total digit count; scale is the fractional digit count, including zero. Either position may be null when unknown; Rosetta omits the annotation when both are unknown. Dynamic decimal columns default missing or null dimensions to precision 15 and scale 2. |
StructureItem::field($source) removes x-ferry-trans, title and
description before producing the logical
schema returned to callers. Converter and numeric annotations remain.
Dynamic decimal creation reads precision and scale from the effective output schema, defaulting to 15 and 2 when absent.
Example of a stored StructureItem.schema:
{
"type": "object",
"x-ferry-primary": ["_idrref"],
"required": ["_idrref"],
"properties": {
"_idrref": {
"type": "string",
"format": "uuid",
"minLength": 36,
"maxLength": 36,
"x-ferry-converter": "uuid"
},
"_code": {
"type": "string",
"title": "Code",
"x-ferry-trans": "phpinnacle-ferry::resources.replica.system_fields._code"
},
"_fld17_rrref": {
"type": "string",
"format": "uuid",
"minLength": 36,
"maxLength": 36,
"title": "Owner",
"x-ferry-converter": "uuid"
},
"_fld18_n": {
"type": "number",
"title": "Amount",
"minimum": 0,
"exclusiveMaximum": 100000000,
"multipleOf": 0.0001,
"x-ferry-decimal": [12, 4]
}
}
}
The item's table name, ordered replication primary_key array and priority are model attributes.
For reusable presentation annotations such as
x-icon, x-color, x-badges and x-enum, see
Eidos's extension reference.
Transform labels, descriptions and form parameters use phpinnacle-ferry::transforms.* translation keys from resources/lang/{locale}/transforms.php.
Each transformation implements PHPinnacle\Ferry\Contracts\Transform and owns its editor block, input compatibility, output JSON Schema and Kafka Connect configuration. Configure its class in phpinnacle-ferry.kafka.transforms; FerryServiceProvider resolves each class through Laravel's container and registers the resulting objects in the shared TransformRegistry. Forms and workers use the same instances, keyed by getKey().
The default list contains DebeziumTimestamps, Unwrap, Cast, Filter, InsertUuid, Slug, TimestampConverter, BooleanToString, StringMap, EmptyStringToNull, Toggle, Json, Money and Radix. Add the optional string transforms to this list as needed:
// kafka.transforms in config/phpinnacle-ferry.php
[
\PHPinnacle\Ferry\Transformations\DebeziumTimestamps::class,
\PHPinnacle\Ferry\Transformations\Unwrap::class,
\PHPinnacle\Ferry\Transformations\Cast::class,
\PHPinnacle\Ferry\Transformations\Filter::class,
\PHPinnacle\Ferry\Transformations\InsertUuid::class,
\PHPinnacle\Ferry\Transformations\Slug::class,
\PHPinnacle\Ferry\Transformations\TimestampConverter::class,
\PHPinnacle\Ferry\Transformations\BooleanToString::class,
\PHPinnacle\Ferry\Transformations\StringMap::class,
\PHPinnacle\Ferry\Transformations\EmptyStringToNull::class,
\PHPinnacle\Ferry\Transformations\Toggle::class,
\PHPinnacle\Ferry\Transformations\Json::class,
\PHPinnacle\Ferry\Transformations\Money::class,
\PHPinnacle\Ferry\Transformations\Radix::class,
\PHPinnacle\Ferry\Transformations\Trim::class,
\PHPinnacle\Ferry\Transformations\Substring::class,
\PHPinnacle\Ferry\Transformations\MaskField::class,
];
Json serializes a selected field to a JSON string without a schema/payload envelope, preserving nulls. Strings become quoted JSON strings; arrays and objects are supported. Kafka Connect supplies the encoding, including its native logical-type and Base64 byte encoding. Ferry uses field mode so the enclosing record remains available to JDBC.
EmptyStringToNull replaces the exact empty string ("") with null. Its trim toggle defaults to false; when enabled, leading and trailing whitespace is removed from every selected string before checking for emptiness, so " " becomes null and " text " becomes "text". Existing nulls and tombstones are preserved. The output JSON Schema is string|null, without source defaults, formats or constraints; static destinations must accept nullable plain text. It supports source fields and string constants and emits org.phpinnacle.toblerone.EmptyStringToNullTransform$Key or $Value before renames. Install Toblerone v0.0.11 or later on every Connect worker. Ferry stores the native Boolean trim setting and does not execute string conversion in PHP.
Money converts Decimal or FLOAT64 amounts to INT64 currency minor units using the selected uppercase ISO currency code: RUB 1.2300 becomes 123, JPY 123.0000 becomes 123, and KWD 1.2340 becomes 1234. Numeric schemas, including scale-zero numeric columns represented as integers, are offered by the editor. The actual Kafka value must be Decimal (BigDecimal) or FLOAT64 (Double); use Debezium's decimal.handling.mode=precise or double with compatible serializers. FLOAT64 requires Toblerone v0.0.12 or later and also supports Ferry's floating-point constants and Cast(float64) → Money chains. Integer and FLOAT32 values remain unsupported; integer constants need an explicit Cast to float64 first. Double values use their canonical decimal representation, so BYN 0.29 becomes 29. Precision lost upstream cannot be recovered; use precise when the original decimal precision matters. Currency support and fraction digits come from Toblerone's ICU data. Fractional minor units, non-finite values and INT64 overflow fail the task; values are never rounded or truncated. Floating-point arithmetic artifacts such as 0.1 + 0.2 can therefore fail conversion.
Radix converts integer values to strings and parses strings as signed 32-bit integers in bases 2 through 36. Temporal fields are excluded. Invalid digits and string values outside the signed 32-bit range fail the task. Integer input may be 64-bit, so reversing that conversion is not guaranteed for larger values. All three transforms preserve nullability, discard obsolete input constraints and support both key and value targets, including tombstones. Install the corresponding Toblerone classes on the Kafka Connect worker.
DebeziumTimestamps (debezium_timestamps) and Unwrap (unwrap) are registered record transforms available in the upstream form. Timestamp settings are created_field and updated_field, defaulting to created_at and updated_at; the names must differ. Unwrap's delete_mode defaults to tombstone. Both require the Debezium envelope, so place timestamps before unwrap and apply unwrap only once. Although the shared record catalog also offers them in replica forms, they must run before any sink unwrapping.
For an upstream pipeline, save native rows or typed TransformSetting objects:
$upstream->update(['transforms' => [
['type' => 'debezium_timestamps', 'data' => [
'created_field' => 'created_at',
'updated_field' => 'updated_at',
]],
['type' => 'unwrap', 'data' => ['delete_mode' => 'tombstone']],
]]);
This unwrap example is for flat-row JDBC topics. Leave unwrap out of the source pipeline for Eloquent change events. The equivalent origin-wide configuration is:
'transforms' => [
'timestamps' => [
'type' => 'org.phpinnacle.toblerone.DebeziumTimestampsTransform',
'created.field' => 'created_at',
'updated.field' => 'updated_at',
],
'unwrap' => [
'type' => 'io.debezium.transforms.ExtractNewRecordState',
'delete.tombstone.handling.mode' => 'tombstone',
],
],
Toblerone's timestamp SMT fills missing/null creation and update timestamps on creates and snapshot reads, and only update timestamps on updates. Existing values win; deletes and tombstones remain unchanged. Fallback time is the envelope's ts_ms, or the worker clock when absent, rather than the original database creation time. Newly added schemaful fields are optional Kafka Timestamp values. Install the Toblerone JAR containing DebeziumTimestampsTransform on every Connect worker. Source transforms do not add properties to Ferry's saved structure; declare generated fields in the replica's input schema before mapping them.
Transform::block() returns InlineBlock|TabbedBlock, using aliases for Common's InlineBuilder\Block and TabbedBuilder\Block. Field transforms return an InlineBlock; built-in record transforms return Ferry's RecordTransformBlock, a TabbedBlock subclass. Each call creates an independent block with its own label, icon and parameter form. A block's name must match the transformation's getKey(). Field parameter schema closures receive the preceding ?Schema $input, the owning FieldTransforms $component and optional ?int $index. The input is null when a preceding transformation has no valid output. Field forms are built when opened, and their components are cloned for each instance. Record tabs use the native Filament builder lifecycle. For dependent record fields, the owning RecordTransforms::getInputSchema($itemKey) returns the schema before that item. Blank or partial live state is represented by a null transform output; the preview keeps the preceding valid schema.
RecordTransformBlock::parameters() accepts a callback such as fn (Schema $input) => [...]. It supplies the schema before the current record transform, including preceding transform settings, without requiring the transform to navigate the component hierarchy. Use $block->getInputSchema() to read the current record preview in availability callbacks. Both transformation editors receive schemas without a StructureItem model. Native TabbedBlock schemas remain supported for record transforms that do not need this context.
Implement the same contract for an application transform. This example uses Kafka's native masking transform:
use Filament\Forms\Components\TextInput;
use PHPinnacle\Common\Forms\InlineBuilder\Block as InlineBlock;
use PHPinnacle\Jason\Schema;
use PHPinnacle\Ferry\Contracts\Transform;
use PHPinnacle\Ferry\Enums\TransformTarget;
use PHPinnacle\Ferry\Enums\TransformScope;
final class Redact implements Transform
{
public function getKey(): string
{
return 'redact';
}
public function getScope(): TransformScope
{
return TransformScope::Field;
}
public function sourceFields(array $params): array
{
return [];
}
public function accepts(Schema $input): bool
{
return $input->singleType() === 'string';
}
/** @param array{replacement: string} $params */
public function output(Schema $input, array $params): Schema
{
return Schema::fromArray([
'type' => in_array('null', $input->types(), true) ? ['string', 'null'] : 'string',
]);
}
public function block(): InlineBlock
{
return InlineBlock::make($this->getKey())
->label('Redact')
->schema(fn () => [
TextInput::make('replacement')->default('***')->rules(['string'])->required(),
]);
}
/**
* @param array{field: string, replacement: string} $params
* @return array<string, string>
*/
public function connector(TransformTarget $target, array $params): array
{
return [
'type' => 'org.apache.kafka.connect.transforms.MaskField$' . ucfirst($target->value),
'fields' => $params['field'],
'replacement' => $params['replacement'],
...($target !== TransformTarget::Key ? ['predicate' => 'tombstone', 'negate' => 'true'] : []),
];
}
}
// Add Redact::class to kafka.transforms in config/phpinnacle-ferry.php.
accepts() receives the complete preceding JSON Schema. output() returns the resulting Schema, or null when settings are incomplete and an output cannot yet be determined. Return the input unchanged only when the transform preserves all its constraints. Each transform declares its parameter array shape and validates it through the block's native Filament form. Parameters can contain structured values such as replacement dictionaries; saved settings preserve these arrays. FieldTransforms uses the forms' native dehydration for schema previews and reordering as well as submission. getScope() declares TransformScope::Field or TransformScope::Record. sourceFields() lists physical or previously generated input dependencies for record transforms and returns an empty list for field transforms. connector() receives a required TransformTarget $target followed by the saved parameters; it produces the corresponding native key or value transform. The caller selects the target for each execution. Key transformations must run on tombstones without reading the value; for field transforms, the factory binds the owning field as the reserved field parameter and returns a flat Kafka Connect configuration for one transform. The pipeline factory assigns its alias. Configured transformations are available without booting a Filament panel and must not retain request or form state.
FieldTransforms filters its block catalog by field scope. getAvailableBlocks() then filters by native block availability and the current chain's output schema. Registered field blocks remain available for editing when the chain no longer accepts them as additions.
TransformRegistry::find($key) returns a registered transform or null; get($key) throws when the key is missing, and all() returns the catalog. ReplicaSchemaResolver::resultSchema(Schema $input, array $settings, TransformScope $scope = TransformScope::Field) accepts a list of Data\TransformSetting objects, passes the schema through each transformation and returns null for an incompatible scope, input or unavailable output. Schema resolution does not depend on the Kafka key/value target. Each step consumes the preceding output. Form and storage boundaries, including FieldTransforms::getSettings(), use ordered type/data arrays; convert those rows with TransformSetting::fromArray() before passing them to the schema resolver.
Cast replaces value constraints with the selected scalar type and preserves nullability; repeated casts use that result. Trim, Substring and Mask accept strings, including formatted and nullable strings, but reject date, time and date-time. UUID strings, including reference identifiers, remain supported. Their outputs retain nullability and discard the source format, enum, pattern and length bounds.
Forms and previews use accepts(), output() and block() without calling connector(). ReplicaSchemaResolver::mappingSchema($mapping, $inputSchema) reads the source property from the supplied effective record schema, or derives a schema from the native static value, then applies the mapping's ordered field transforms. It returns null when the record schema is unavailable. ReplicaSchemaResolver::outputSchema($replica, $mapping) resolves the replica's effective record schema; use mappingSchema() with an already resolved schema, including original and current schemas when planning table changes. FieldMapping::canMapTo($output, $target) checks destination compatibility. A mapped output must have a supported scalar type. ReplicaTableManager translates it into SQL columns; decimal precision and scale come from the output schema.
Origins\Kafka\TransformFactory::pipeline($replica, $object, keyMap: [], fixedValues: []) uses the shared registry's connector() implementations and returns the complete flat Kafka Connect configuration: ordered aliases, transforms.* properties and required tombstone predicates. It applies replica.transforms to the value in list order and also runs the selected key dependencies against the key, then source and constant transforms in mapping-row order, then destination fixed values, and finally key/value renames. Static and fixed fields use temporary names until the final value projection, preventing collisions with physical fields; destination fixed values take precedence. The final projection includes only mapped and fixed fields. Tombstones pass through to the JDBC sink so both static and dynamic destinations propagate deletions. Pass the source-to-destination key map to retain and rename its source key fields using Apache ReplaceField. Both sink modes pass the key map in destination primary-key order. Selected generated keys are computed from physical record-key dependencies before renaming, including on tombstones. Each record transform uses the explicit Value target; selected key dependencies also use the explicit Key target. Generated fields can feed subsequent transforms on either path. The default empty key map omits key selection and renaming. A missing transformation fails assembly through the registry's transform_missing error.
Both destination types allow compatible field transforms on physical and generated key fields. The user is responsible for uniqueness of transformed keys. Transforms for fields mapped to destination keys operate on the record key; ordinary and constant fields use the value. Filter continues to skip tombstones so it cannot suppress destination deletes.
Field transformations use Common's InlineBuilder; record transformations use its TabbedBuilder. Run php artisan filament:assets to publish the shared editor assets.
Transform settings are ordered ['type' => 'mask', 'data' => ['replacement' => '***']] entries; data retains the native parameter types and may be omitted when empty. Each mapping row stores its ordered field transforms under options['transforms']. Native Filament forms validate parameters and type chains; ReplicaSchema checks destination compatibility before saving form state. Connector assembly consumes the saved mapping without repeating form validation. Saving a configured replica refreshes its desired sink configuration; the shared source connector changes only when its generated configuration changes. Kafka compiles the source and sink configurations before saving either one. A missing transformation raises the registry’s transform_missing error during activation or update; there is no automatic pause for configuration changes.
Replica.transforms exposes an ordered list<TransformSetting> for record-level settings stored in a JSON column, defaulting to an empty list. The record editor submits native type/data rows; TransformsCast accepts those rows or typed settings and restores typed settings when reading the model. Model serialization returns the native rows. These settings determine the effective record schema, source capture dependencies, and the first steps of the connector pipeline.
Upstream.transforms uses the same cast and ordered rows for the source connector. The upstream editor validates against a generic object schema because captured tables can have different columns; it does not provide table-specific field choices. Saved settings run as upstream_0, upstream_1, and so on, after origin-wide source transforms and before serialization, for every captured table. Saving upstream transforms regenerates the desired source configuration without restarting structure preparation or changing replica transforms.
The built-in Cast transformation targets the native Kafka Connect Cast types: string, int8, int16, int32, int64, float32, float64, and boolean. It accepts unions of scalar types, including null, and rejects arrays and objects. Integer targets produce JSON Schema integer; floating-point targets produce number. The Kafka adapter uses the selected target directly. Enums\KafkaType defines these values and their JSON Schema types. Its HasLabel implementation supplies translated selector labels with explicit numeric widths; saved parameters and Kafka configurations keep the native string values. The selector disables the current unformatted string or boolean type using the output of preceding transforms, including nullable inputs. Numeric widths remain available because JSON Schema does not track the Kafka width. Formatted strings, unknown inputs and unions with several non-null types keep every target available. Casting removes the input format and constraints and preserves nullability. Choose numeric widths for the values being replicated: narrowing integers can overflow and float32 reduces precision.
Replace null with default controls Kafka's replace.null.with.default: it defaults to Yes and uses the default declared in the field schema, preserving null when no default exists. Choosing No stores replace_null_with_default: "false"; the default Yes remains implicit. The output schema remains nullable when the input is nullable, because a Kafka field default is not guaranteed.
TimestampConverter (timestamp_converter) uses Apache Kafka's native TimestampConverter on the selected field. It accepts text, integer epoch values and logical date/time inputs, including nullable values, and rejects booleans, floating-point numbers, arrays and objects. Settings are target_type (string, unix, Date, Time, Timestamp), optional format, unix_precision (seconds, milliseconds, microseconds, nanoseconds; default milliseconds) and optional replace_null_with_default (the same Kafka default behavior as Cast). The editor requires a Java SimpleDateFormat pattern for plain text input or string output; Kafka Connect validates the pattern and streamed values. Formatted output has a plain string schema because arbitrary Java patterns do not guarantee a JSON Schema date format. Logical Date, Time and Timestamp outputs use date, time and date-time; Unix output is an integer. Every output discards the previous constraints and preserves nullability. Text transforms cannot follow logical date/time outputs without an explicit conversion to string.
Conversion runs before mapping renames and skips tombstones, preserving JDBC deletes. Unix precision describes the units of integer input or output; it is not inferred from Ferry's JSON Schema. Check the actual Connect schema: native logical Date/Time/Timestamp values are handled as logical types, while Debezium integer timestamp representations require the matching Unix precision. Text representations with a temporal JSON Schema format still need an explicit parsing pattern. Kafka's SMT uses UTC for text formatting and loses sub-millisecond precision during conversion. It does not correct source timezone assumptions or 1C-specific date offsets. Ferry emits connector configuration and does not parse timestamps in PHP.
BooleanToString (boolean_to_string) converts boolean fields to the text configured in true and false. Its native text inputs preserve empty strings and whitespace. The output is a string enum of the possible replacements with their minimum and maximum lengths; nullable inputs keep null, and input enum restrictions narrow the output. Toggle (toggle) inverts boolean values with no settings, retains nullability and inverts any effective enum restrictions. Both accept only boolean fields, including nullable booleans.
StringMap (string_map) uses a native Filament KeyValue field named mapping. For example, ['mapping' => ['draft' => 'Draft', 'deleted' => '']] compiles to field=<source>, mapping.draft=Draft and mapping.deleted=. Matching is exact, case-sensitive and preserves whitespace; replacements are applied once and unmatched values remain unchanged. Empty replacement values are allowed. Native KeyValue omits blank keys, so the editor cannot configure a replacement for an empty source string. At least one mapping is required and every replacement must be a string. StringMap accepts ordinary strings, including formatted strings, but excludes logical date/time fields. Its output preserves nullability, remaps finite enums (including unmatched values), and drops obsolete formats and patterns. Finite enums produce exact output length bounds; otherwise, a known source maximum length is widened to include all replacements.
These transforms run on the selected value field before renames, also support replica constants, and preserve tombstones through Toblerone's native behavior. Install Toblerone v0.0.5 or later on every Connect worker. Ferry emits native org.phpinnacle.toblerone.BooleanToStringTransform$Value, StringMapTransform$Value and ToggleTransform$Value configurations; streamed values are transformed on the worker, not in PHP.
Static values use rows with kind: static, a required target, and a native string, boolean, integer or float in static. Their type comes from the native value; 1.0 stays a float through JSON serialization. They compile to InsertField plus a cast when the native value requires it. Optional options['transforms'] run next on the temporary field, before the final projection renames it to target; their output JSON Schema is calculated from the native value and the registered definitions. No separate source name is stored in the mapping row.
Register the optional Trim, Substring, and Mask implementations as shown above. The mapping supplies the selected source field; administrators do not enter a second field name.
| Key | Settings form | Kafka Connect class |
|---|---|---|
trim |
No settings; trims whitespace from both ends of the selected string. | org.phpinnacle.toblerone.TrimTransform$Value |
substring |
Required zero-based start, optional exclusive end. A blank end is omitted; zero is retained. |
org.phpinnacle.toblerone.SubstringTransform$Value |
mask |
Required replacement, default ***, replacing the entire string. |
org.apache.kafka.connect.transforms.MaskField$Value |
Trim requires the Toblerone plugin and preserves null field values and tombstones. Substring serializes indexes as fields=source:start[:end] and requires the Toblerone plugin on the worker. It uses Java UTF-16 indexes without clamping: indexes must fit every input string. Masking uses Apache's fields and replacement settings and skips tombstones through the built-in predicate. Other MaskField options keep their Kafka defaults. Forms validate the settings; Ferry does not execute the transformations in PHP.
Filter keeps (include) or drops (exclude) entire records using a JSONPath predicate scoped to the owning source field, including sources without a target. Enter the predicate body: @ == true, @ >= 100, or @.price > 100. @ denotes the selected field; $ also refers to that field as the local root. Nested [?()] filters retain their @ item scope. Quoted values and regular expressions are preserved. For example, @ >= 100 on _amount becomes $[?(@['_amount'] >= 100)]. Numbers remain literal text, preserving decimal precision.
The predicate runs at its position in the ordered chain, so preceding Trim or Cast transforms affect the value it reads. Filter accepts every input type and preserves its schema. The condition placeholder and hint follow its effective input type: string equality, numeric comparison, Boolean equality, nonempty arrays, or an object child comparison. Nullable types use their non-null type; null-only inputs use @ == null, and unknown or mixed inputs use @ != null. Changing the input type updates these examples without changing the saved predicate.
The form stores condition, mode, and missing_or_null_behavior (include, exclude, or fail; default include). These settings configure Confluent's Filter$Value. Kafka Connect checks JSONPath syntax; Ferry validates the required expression and enum settings. Confluent applies missing/null behavior to paths outside predicates; missing or null references inside [?()] are treated as undefined and do not match. Ferry emits a predicate, so those native predicate semantics apply. Filters skip tombstones through the negated RecordIsTombstone predicate, preserving Debezium deletions.
The source connector captures fields with a target or transforms; unbound filter fields are excluded from destination writes by the sink whitelist.
RecordFilter (record_filter) is available in both upstream and replica record transformation editors. It uses the same action and missing/null settings as Filter, preserves the record schema and skips tombstones. Enter a complete JSONPath expression, such as $[?(@.amount >= 100 && @.active == true)] or $.after.customer[?(@.address.city == 'Warsaw')]. Ferry passes it unchanged to Confluent's Filter SMT without adding a predicate wrapper or binding it to a field. Paths refer to the record at that position in the pipeline: before Unwrap they address the Debezium envelope; after Unwrap they address row fields. Missing/null behavior applies to paths outside predicates; references inside [?()] follow the predicate semantics described above. Kafka Connect validates JSONPath syntax.
Implementation limitation: record filter expressions do not contribute column dependencies to source capture. With Rosetta, filtering on a column does not cause it to be captured; it must already be included through replica mappings or another transform's declared dependencies. This applies to both upstream and replica record filters. Eloquent captures all columns of its selected model tables. Record filters only pass or drop incoming messages: when an existing destination row stops matching, dropping its update does not remove that row.
For the Ferry transformation editor, use the standalone FieldTransforms field with a source object schema represented by Schema:
use PHPinnacle\Ferry\Forms\FieldTransforms;
->sourceSettings(fn (string $source) => [
FieldTransforms::make('transforms')
->label($source)
->inputSchema($sourceSchema->properties()[$source]),
])
Replica forms allow compatible field transforms on every source, including physical and generated keys. Generated keys are produced by record transforms. FieldTransforms::inputSchema(Schema|Closure) takes the source property schema after record transforms. A schema callback must return Schema and supports native Filament utility injection. RecordTransforms::inputSchema(Schema) takes the fixed record input schema. Configure the input schema before using either editor; arrays belong at serialization boundaries. The field passes this complete schema through ReplicaSchemaResolver::resultSchema() to determine compatible additions. Mapping consumers use the same result in their outputSchema() callbacks for hints and connection checks. For FieldBinding static source settings, pass $inputSchema directly to `FieldTr
phpinnacle/ferry is a public repo on GitHub by phpinnacle. A plugin for Laravel Filament that manages data source connections.
It declares the MIT license on GitHub.
Its maker has not tagged a system, so it is unknown here. Its README says what it needs.
Open "Take it" on its page: Take My Repo writes an install plan for your machine and hands it to your AI agent (Claude Code, Codex or Cursor), which asks you before each step. Take My Repo has not tested it on any machine.
Its last change on GitHub was on 2026-11-18. Take My Repo does not judge from that whether it is maintained.
Nobody has reviewed it on Take My Repo yet, which says nothing about its quality.
Simple library for operating binary data in PHP
Pure PHP asynchronous implementation of the Cassandra V4 binary protocol.