Command-line functions

Many of the functions of followthemoney (FtM) can be used interactively or in scripts via the command line. Please first refer to the Aleph documentation for an intro to the ftm utility.

Key to understanding the ftm tool is the notion of Streams: entities can be transferred between programs and processing steps as a series of JSON objects, one per line. This notion is supported by the related alephclient command, which can serve as a source, and a sink for entity streams, backed by the Aleph API.

Examples

The command line sequence below uses shell pipes to a) map data into entities from a database, b) apply a namespace to the entity IDs, c) aggregate entity fragments created by the mapping, and d) export the resulting entity stream into a sequence of CYPHER statements that can be executed on a Neo4J database to generate a property graph:

ftm map companies_from_db.yml | \
    ftm sign -s my_namespace | \
    ftm aggregate | \
    ftm export-cypher -o graph.cypher

Here’s another example that fetches pre-generated entities from a URL and loads them into a local Aleph instance:

export URL=https://public.data.occrp.org/datasets/icij/panama_papers.ijson
curl -s $URL | \
    ftm validate | \
    alephclient write-entities -f icij_panama_papers

Command reference

ftm

Utility for FollowTheMoney graph data

ftm [OPTIONS] COMMAND [ARGS]...

aggregate

Aggregate multiple fragments of entities

ftm aggregate [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>

dump-model

Export the current schema model

ftm dump-model [OPTIONS]

Options

-o, --outfile <outfile>

export-csv

Export to CSV

ftm export-csv [OPTIONS]

Options

-i, --infile <infile>
-o, --outdir <outdir>

output directory

export-cypher

Export to Cypher script

ftm export-cypher [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
-e, --edge-types <edge_types>

Property types to be reified into graph edges.

Options

iban | identifier | country | url | domain | date | ip | phone | checksum | entity | address | name | email

export-excel

Export to Excel

ftm export-excel [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>

Required

export-gexf

Export to GEXF (Gephi) format

ftm export-gexf [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
-e, --edge-types <edge_types>

Property types to be reified into graph edges.

Options

iban | identifier | country | url | domain | date | ip | phone | checksum | entity | address | name | email

export-neo4j-bulk

Export to Neo4J bulk import

ftm export-neo4j-bulk [OPTIONS]

Options

-i, --infile <infile>
-o, --outdir <outdir>

output directory

-e, --edge-types <edge_types>

Property types to be reified into graph edges.

Options

iban | identifier | country | url | domain | date | ip | phone | checksum | entity | address | name | email

export-rdf

Export to RDF NTriples

ftm export-rdf [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
--qualified, --unqualified

Generate full predicates

import-vis

Load a .VIS file and get entities

ftm import-vis [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>

map

Execute a mapping file and emit objects

ftm map [OPTIONS] MAPPING_YAML

Options

-o, --outfile <outfile>
--sign, --no-sign

Apply HMAC signature

Arguments

MAPPING_YAML

Required argument

map-csv

Map CSV data from stdin and emit objects

ftm map-csv [OPTIONS] MAPPING_YAML

Options

-i, --infile <infile>
-o, --outfile <outfile>
--sign, --no-sign

Apply HMAC signature

Arguments

MAPPING_YAML

Required argument

match-decide

Generate match decisions based purely on score

ftm match-decide [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
-t, --threshold <threshold>

match-entities

Unnests matches into entities

ftm match-entities [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
-a, --all

Unnest non-positive matches

pretty

Format a stream of entities to make it readable

ftm pretty [OPTIONS]

Options

-i, --infile <infile>

sieve

Filter out parts of entities.

ftm sieve [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
-s, --schema <schema>

Filter out the given schemata.

Options

Document | Workbook | PlainText | Table | Package | Family | ProjectParticipant | Company | LegalEntity | Assessment | CourtCase | Person | Article | Pages | Debt | Folder | Interest | BankAccount | Message | License | Page | Sanction | Interval | Event | Documentation | Organization | CryptoWallet | Airplane | Image | Representation | Vessel | Employment | Asset | Thing | Vehicle | Succession | Value | Payment | Identification | Analyzable | CourtCaseParty | Security | Passport | RealEstate | Project | Video | Ownership | Audio | Call | ContractAward | UnknownLink | EconomicActivity | Address | Contract | TaxRoll | Mention | PublicBody | Email | Associate | HyperText | Directorship | Membership | UserAccount | Note

-p, --property <property>

Filter out the given property names.

-t, --type <type>

Filter out the given property types.

Options

iban | number | country | date | ip | text | address | name | mimetype | url | json | string | entity | phone | language | identifier | domain | html | checksum | topic | email

sign

Apply a HMAC signature to entity IDs

ftm sign [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>
-s, --signature <signature>

HMAC signature key

validate

Re-parse and validate the given data

ftm validate [OPTIONS]

Options

-i, --infile <infile>
-o, --outfile <outfile>