=begin rakudoc

=TITLE Adding Sources/Destinations

=SUBTITLE Connecting Qwiratry pipelines to files, databases, APIs, and other endpoints

=head1 Overview

Sources and destinations connect Qwiratry pipelines to external locations.

Use this extension point when the new thing is a place Qwiratry receives data
from or outputs data to: files, Postgres, SQLite, HTTP APIs, message queues,
object stores, search indexes, and similar systems.

If the work is only about converting between external data representations and
Qwiratry-visible data, use a format extension instead.

To add support for a new source or destination:

=item 1. B<Setup>: decide whether the extension receives data, outputs data, or does both.

=item 2. B<Backend module>: create a location backend module under C<Qwiratry::Location>.

=item 3. B<Source>: define how Qwiratry receives data from the external location.

=item 4. B<Destination>: define how Qwiratry outputs data to the external location.

=item 5. B<Testing>: test validation, receive behavior, output behavior, errors, and pipeline composition.

=head1 Setup

Decide whether the extension is source/destination behavior or a format. A
database, API, queue, or object store is a source/destination even if it also
uses a format internally.

Define validation rules for locations, connection strings, credentials, or other
connection details. Decide whether the extension returns data, rows, streams, or
another structured value.

Location backends are loaded through L<Qwiratry::Location>. The backend name is
inferred from the location scheme: C<postgres://example/db> resolves to
C<Qwiratry::Location::Postgres>, C<s3://bucket/key> resolves to
C<Qwiratry::Location::S3>, and local paths or C<file://> URLs resolve to the
built-in C<Qwiratry::Location::File> backend.

=head1 Backend Module

A backend module groups the source and destination implementations for one
location family. A backend can implement only source, only destination, or both.

=begin code :lang<raku>
use X::Qwiratry;
use Qwiratry::Location::Base;

class Qwiratry::Location::Postgres::Source is Qwiratry::Location::Base::SourceWalkable {
    method read-walkable(Str $location --> Str) {
        # Connect to $location and return text, rows encoded as text,
        # or another representation expected by the rest of the pipeline.
    }
}

class Qwiratry::Location::Postgres::Destination is Qwiratry::Location::Base::DestinationWalkable {
    method write-walkable(Str $location, Mu $content --> Mu) {
        # Connect to $location and write $content.
        $content
    }
}
=end code

No registration step is required. Once the module is available as
C<Qwiratry::Location::Postgres>, the factory can discover
C<Qwiratry::Location::Postgres::Source> and
C<Qwiratry::Location::Postgres::Destination>.

=head1 Source

=head2 Source Responsibilities

=item Validate the location or connection details early.

=item Receive data from the external system without changing query semantics.

=item Return data in a form that later operators can parse, walk, or transform.

=item Throw C<X::Qwiratry::IO::LocationError> for location failures.

=head2 Minimal File Source Example

This example uses C<SourceOperator> to receive file data, ready to be passed to
a parser.

=begin code :lang<raku>
use Qwiratry::Operator::IO;
use Qwiratry::Operator::Capability;

my $source = SourceOperator.new(:location<input.csv>);
my $data = execute($source);
=end code

Under the operator surface, Qwiratry resolves the concrete source through:

=begin code :lang<raku>
Qwiratry::Location.make(:type<Source>, :location<input.csv>).read-walkable('input.csv')
=end code

C<SourceOperator.evaluate> always uses C<read-walkable> (it does not return an
C<Iterator>). Directory and glob locations keep their walkable return shapes.

=head2 Optional pull (C<SourcePull> / C<TypedIterator>)

When a backend also does C<Qwiratry::Location::Base::SourcePull>, it may
implement C<read-pull>, which SHOULD return a L<TypedIterator> whose C<.of>
matches the element kind (File uses C<Str> for text chunks).

Pull vs walkable is discovered with C<~~ SourcePull> (and C<~~ DestinationPush>
on the other edge). Element kind lives on the iterator via C<.of>, not on a
separate Location API.

C<ParseOperator> composes C<read-pull> + C<parse-pull> when both the source
backend and the format support pull. Text adaptors reject non-text C<.of> and
non-C<Str> elements when C<.of> is C<Mu>
(C<X::Qwiratry::IO::ElementType>). Structured (row) C<.of> and row sinks are
TOP Basics work, not required for blob backends.

=head1 Destination

=head2 Destination Responsibilities

=item Validate the destination location or connection details.

=item Accept rendered content or structured values as appropriate.

=item Preserve pipeline return behavior so callers can observe what was output.

=item Throw C<X::Qwiratry::IO::LocationError> for output failures.

=head2 Minimal File Destination Example

This example uses C<DestinationOperator> to output already-rendered data. The
operator returns the same content it outputs, which keeps pipeline behavior
observable.

=begin code :lang<raku>
use Qwiratry::Operator::IO;
use Qwiratry::Operator::Capability;

my $rendered = "name,score\nalpha,10\nbeta,5\n";
my $destination = DestinationOperator.new(
    :location<output.csv>,
    :subject($rendered),
);

my $output = execute($destination);
=end code

Under the operator surface, Qwiratry resolves the concrete destination through:

=begin code :lang<raku>
Qwiratry::Location.make(:type<Destination>, :location<output.csv>).write-walkable('output.csv', $rendered)
=end code

=head1 Testing

Add tests that prove:

=item Invalid locations or connection strings are rejected.

=item Unavailable locations throw C<X::Qwiratry::IO::LocationError>.

=item Sources receive expected data.

=item Destinations output expected data.

=item C<Qwiratry::Location.backends(:type<Source>)> or
      C<Qwiratry::Location.backends(:type<Destination>)> discovers the backend.

=item Pipelines compose source, parse, query, render, and destination steps correctly.

=head1 Reference Material

=head2 Current Surface

The current source/destination surface is represented by IO operators:

=item C<Qwiratry::Operator::IO::SourceOperator>

Receives data from a location.

=item C<Qwiratry::Operator::IO::DestinationOperator>

Outputs data to a location.

=item C<Qwiratry::Location>

Factory that discovers location backends and creates source or destination
implementation objects.

=item C<Qwiratry::Location::Base>

Defines C<Qwiratry::Location::Base::SourceWalkable> and
C<Qwiratry::Location::Base::DestinationWalkable>, the abstract contracts for backend
implementations.

=item C<Qwiratry::Location::File>

Built-in local file backend. Local paths and C<file://> URLs resolve here.

=item C<X::Qwiratry::IO::LocationError>

Exception for invalid, unavailable, or unsupported locations.

=head2 Backend Shape

Database-like sources should not be implemented as formats. They usually need
connection state, authentication, schema discovery, query pushdown, streaming,
or transaction behavior.

A source/destination extension introduces backend-specific classes such as:

=begin code :lang<raku>
Qwiratry::Location::Postgres::Source
Qwiratry::Location::Postgres::Destination
Qwiratry::Location::HTTP::Source
Qwiratry::Location::S3::Destination
=end code

=head2 Example Direction

A Postgres source is a source/destination extension, not a format extension.
The format layer might still be used after receiving data, but Postgres itself is a
live backend with query and connection semantics.

=begin code :lang<raku>
my $source = SourceOperator.new(:location<postgres://example/db>);
=end code

=end rakudoc
