Configuration Files

Last Updated: 2022-02-09

Grouparoo projects contain configuration files for Apps, Models, Sources, Properties, Groups, Destinations, and any other objects generated in the course of configuration. These config files can be created and updated with UI config.

Config files are generated by the UI as JSON, but can also be converted to JavaScript to manipulate data with more flexibility or load configuration from external sources. JSON files are parsed with JSON5, which enables you to use comments and multi-line strings.

For examples of config files, you can browse app-example-config on Github.

Config Files Location

Config files are saved at the root of your Grouparoo project directory, in a directory called config. A typical Grouparoo project structure looks like this:

my-grouparoo-project/
├── package.json
├── package-lock.json
├── .env
└── config/
    └── ... (config files go here)

You can optionally choose to use another directory for your configuration by setting the GROUPAROO_CONFIG_DIR environment variable, for example GROUPAROO_CONFIG_DIR=/path/to/config.

Common Config Attributes

Every type of object has its own definition but all of them require these attributes:

  • id: (required) A unique identifier for this object.
    • Lowercase letters, numbers, and underscores only
    • No spaces
    • Fewer than 38 characters
    • Must be unique among other objects of the same type
  • class: (required) The type of object that is being defined such as App, Model, Source, Property, Group, Destination
  • name: (required) The display name of this object

Apps

The App configures the connections available to create Sources and Destinations.

  • id: (required) Unique identifier for this app config.
  • name: (required) A display name for the app.
  • class: (required) This must be set to "App".
  • type: (required) The type of the app defined by a plugin. While the type can match the name of the plugin, plugins may offer multiple apps and the names may not match.
  • refresh: App refresh query. See below for details.
  • options: The App configuration options. These vary depending on the type of app.

Example:

{
  "class": "App",
  "id": "data_warehouse",
  "name": "data_warehouse",
  "type": "postgres",
  // App Refresh Query
  "refresh": {
    "query": "..."
    "recurringFrequency": 6000
  }
  // Option vary depending on the type of app
  // Below is an example of the options for the Postgres app
  "options": {
    "host": "localhost",
    "port": 5432,
    "database": "...",
    "schema": "public",
    "user": "...",
    "password": "...",
    "ssl": false,
  }
}

App Refresh Query Configuration

App Refresh Query enables Grouparoo to check your data warehouse for new data, and run all Schedules for that App if it finds any. The following Plugins support App Refresh Query:

  • BigQuery
  • ClickHouse
  • MySQL
  • Postgres
  • Redshift
  • Snowflake
  • SQLite

For the best results, configure your query to return a single row of data. For example, you might set it up as:

  "refresh": {
    "query": "SELECT MAX(updated_at) FROM meta_table"
    "recurringFrequency": 6000
  }

This query would check every minute for a new updated_at value in a table called meta_table. The table you check with your App Refresh Query may or may not be one that you include as a Source.

If there are Schedules on this App's Sources that you do not want to be run, you can add "refreshEnabled": false to your Schedule's config.

Models

The Model config file is currently very basic:

  • class: (required) Should be set to "Model".
  • id: (required) The unique value that identifies this Model.
  • name: (required) A display name for the Model.
  • type: (required) Can be set to "profile", "account", or "custom".

Example:

{
  "class": "Model",
  "id": "users",
  "name": "Users",
  "type": "profile"
}

Sources

Each Source has a common set of attributes. Certain types of Sources may have additional attributes, but all Sources will have the following common attributes:

  • class: (required) Should be set to "Source".
  • id: (required) The unique value that identifies this Source.
  • name: (required) A display name for the Source.
  • type: (required) The type of Source will vary based on the Generator you're using. You likely don't want to change this value.
  • appId: (required) The id of the App (connection) that the Source uses.

Example:

{
  "class": "Source",
  "id": "demo_users",
  "modelId": "users",
  "name": "Product Users",
  "type": "postgres-import-table",
  "appId": "demo_db",
  // Options depend on the type of app
  "options": {
    "table": "users"
  },
  // Mapping settings
  "mapping": {
    "id": "user_id"
  }
}

Schedules

A Schedule tells Grouparoo how frequently to check the Source for updated data and import it into the Application Database. This is the most common way to keep data fresh in Grouparoo.

Here are the attributes for a Schedule:

  • id: (required) Like the Source, a Schedule must have a unique id to identify it.
  • name: (required) A display name. It matches the id by default.
  • class: (required) This must be set to "Schedule".
  • sourceId: (required) This must match the id of the Source. It will be set automatically.
  • recurring: (required, default: true) Should this Schedule run regularly?
  • incremental: (default: true) Should this schedule only update records that have changed since the last run?
  • refreshEnabled: (default: false) Should this schedule also trigger when the App Refresh query finds new data?
  • recurringFrequency: (required) The length of the Schedule's interval. This is measured in ms. For example, 15 minutes would be 1000 * 60 * 15 or 900000.
  • options: These are the custom options that the specific Source type will bring.
  • filters: These can be used to exclude certain rows of data from your Schedule. Depending on your Source's database, only some of these filters may be available.

Filters Ops

  • "exists" - exists with any value (is not null)
  • "notExists" - does not exist with any value (is null)
  • "eq"- equals
  • "ne" - does not equal
  • "gt" - greater than
  • "gte" - greater than or equal to
  • "lt" - less than
  • "lte" - less than or equal to
  • "substring" - contains
  • "notSubstring" - does not contain
  • "in" - in a given set

For example, to create a schedule that will run every 5 minutes checking the "updated_at" column for rows where status = "active" and last_purchase" exists (is not null), you could do the following:

{
  "class": "Schedule",
  "id": "users_schedule",
  "name": "Users Schedule",
  "sourceId": "users",
  "recurring": true,
  "recurringFrequency": 5 * 60 * 1000,
  "confirmRecords": false,
  "options": {
    "column": "updated_at"
  },
  "filters": [
    {
      "key": "status",
      "op": "eq",
      "match": "active"
    },
    {
      "key": "last_purchase",
      "op": "exists"
    }
  ]
}

If you've configured an App Refresh Query when you configured your App, you can also opt to leave a particular Schedule out of its updates by adding "refreshEnabled": false to the Schedule's config.

Properties

Each Property has a common set of attributes, while other attributes are specific to the type of Property. Here are the common attributes:

  • class: (required) Should be set to "Property".
  • id: (required) The unique value that defines this Property.
  • isArray: (required) Is the Property an array?
  • name: (required) A display name for the Property. Will be set to the id by default.
  • sourceId: (required) The id of the Source that the Property is created from.
  • type: (required) The data type of the Property.
  • unique: (required) If true, Records will be forced to have unique records for this Property.

Property type depends on the Source App's available properties and may include some or all of these types:

  • boolean
  • date
  • email
  • float
  • integer
  • phoneNumber
  • string
  • url

Example:

{
  "class": "Property",
  "id": "contact_email",
  "type": "email",
  "key": "contactEmail",
  "sourceId": "account_users",
  "unique": false,
  "isArray": false,
  // Options can vary depending on the type of Source
  "options": {
    "aggregationMethod": "least recent value",
    "column": "email",
    "sortColumn": "created_at"
  },
  "filters": []
}

Aggregation Method and Filters

Your Property config file will also contain aggregationMethod and Filter options. Options for these may vary depending on your source type.

Here are some common examples:

aggregationMethods

  • "exact"
  • "sum"
  • "average"
  • "count"
  • "min"
  • "max"
  • "most recent value"
  • "least recent value"

filters

  • "exists" - exists with any value (is not null)
  • "notExists" - does not exist with any value (is null)
  • "eq"- equals
  • "ne" - does not equal
  • "gt" - greater than
  • "gte" - greater than or equal to
  • "lt" - less than
  • "lte" - less than or equal to
  • "substring" - contains
  • "notSubstring" - does not contain
  • "in" - in a given set

These can be combined to create more complex properties such as:

{
  "id": "active_paid_subscriptions",
  "name": "Active Paid Subscriptions",
  "class": "property",
  "sourceId": "users_table",
  "type": "integer",
  "unique": false,
  "identifying": false,
  "isArray": false,
  "options": {
    "column": "subscription_id",
    "aggregationMethod": "count",
    "sort": null,
  },

  "filters": [
    {
      "key": "status",
      "op": "eq",
      "match": "active",
    },
    {
      "key": "last_purchase",
      "op": "exists"
    }
    {
      "key": "subscription_type",
      "op": "ne",
      "match": "trial",
    }
  ]
},

Which would count every subscription per Record, but leave out any trial subscriptions or Records with no value for last_purchase. Or...

{
  "id": "annual_purhcases_2022",
  "name": "Annual Purchases 2022",
  "class": "property",
  "sourceId": "purhcases_table",
  "type": "float",
  "unique": false,
  "identifying": false,
  "isArray": false,
  "options": {
    "column": "purhcase_amount",
    "aggregationMethod": "sum",
    "sort": null
  },

  "filters": [
    {
      "key": "created_at",
      "op": "gt",
      "match": "2021-12-31T11:59:59.999+00.00"
    },
    {
      "key": "created_at",
      "op": "lt",
      "match": "2023-01-01T00:00:00.000+00:00"
    }
  ]
}

...which would return the sum of a Record's purchases in 2022.

Groups

Group config files have a common format.

  • id: (required) The unique value that identifies this Source.
  • class: (required) Should be set to "Group".
  • modelId: (required) The model that the group belongs to.
  • name: (required) A display name for the Group.
  • rules: (required) An array of rules to filter Records. See more details below.

Example:

{
  "id": "high_value_accounts",
  "class": "Group",
  "modelId": "accounts",
  "name": "High Value Accounts",
  "rules": [
    {
      "propertyId": "account_value",
      "op": "gt",
      "match": "800"
    }
  ]
}

Configuring a Group is all about adding Rules to the Group. A Rule is some logical operator to Filter Properties. A Record must match every Rule to be included in the Group.

Available operations depends on both the type of Property and the underlying Grouparoo database (SQLite vs. Postgres). See below for a list of rules. (If in doubt, choose Postgres.)

Note you can use SQL wildcard characters (%, _, [ ], ^, -) in your Group rule match values as appropriate and necessary.

The following operators are available on boolean properties:

exists
exists with any value
{
"propertyId": "subscribed",
"op": "exists"
}
All records that have data about subscription status

notExists
does not exist
{
"propertyId": "subscribed",
"op": "notExists"
}
All records that do not have data about subscription status

eq
is equal to
{
"propertyId": "subscribed",
"op": "eq",
"match": "true"
}
All subscribers

ne
is not equal to
{
"propertyId": "subscribed",
"op": "ne",
"match": "true"
}
All non-subscribers

The following operators are available on date properties:

exists
exists with any value
{
"propertyId": "last_purchase_date",
"op": "exists"
}
All records who have made a purchase

notExists
does not exist
{
"propertyId": "last_purchase_date",
"op": "notExists"
}
All records who have not made a purchase

eq
is equal to
{
"propertyId": "last_purchase_date",
"op": "eq",
"match": "2021-05-25T04:45:00.000+00:00"
}
All records whose most recent purchase was on May 25, 2021 at exactly 4:45AM UTC.

ne
is not equal to
{
"propertyId": "last_purchase_date",
"op": "ne",
"match": "2021-05-25T04:45:00.000+00:00"
}
All records whose most recent purchase was anytime except May 25, 2021 at exactly 4:45AM UTC.

gt
is after
{
"propertyId": "last_purchase_date",
"op": "lt",
"match": "2020-12-25T04:45:00.00+00:00"
}
All records whose most recent purchase is after December 25, 2020 at 04:45:00 UTC.

lt
is before
{
"propertyId": "last_purchase_date",
"op": "lt",
"match": "2021-05-25T09:30:00.000+00:00"
}
All records whose most recent purchase is prior to May 25, 2021 9:30:00 AM UTC.

gte
is on or after
{
"propertyId": "last_purchase_date",
"op": "gte",
"match": "2019-06-14T18:16:00.000+00:00"
}
All records whose most recent purchase is exactly at or after June 14, 2019 at 6:16 PM UTC

lte
is on or before
{
"propertyId": "last_purchase_date",
"op": "lte",
"match": "2020-12-25T04:45:00.000+00:00"
}
All records whose most recent purchase is exactly at or before December 25, 2020 at 04:45:00 UTC.

relative_gt
is in the past
{
"propertyId": "last_purchase_date",
"op": "relative_gt",
"relativeMatchNumber": "60",
"relativeMatchUnit": "days"
}
All records whose most recent purchase is within the last 60 days

relative_lt
is in the future
{
"propertyId": "subscription_ends",
"op": "relative_lt",
"relativeMatchNumber": "30",
"relativeMatchUnit": "days"
}
All records whose subscription ends within the next 30 days

The following operators are available on email properties:

exists
exists with any value
{
"propertyId": "email",
"op": "exists"
}
All records with an email address listed.

notExists
does not exist
{
"propertyId": "email",
"op": "notExists"
}
All records without an email address listed.

eq
is equal to
{
"propertyId": "email",
"op": "eq",
"match": "hello@grouparoo.com"
}
All records with an email address exactly equal to hello@grouparoo.com.

ne
is not equal to
{
"propertyId": "email",
"op": "ne",
"match": "hello@grouparoo.com"
}
All records without an email address exactly equal to hello@grouparoo.com.

like
is like (case sensitive)
{
"propertyId": "email",
"op": "like",
"match": "%smith%"
}
All records with an email address containing smith (case sensitive).

notLike
is not like (case sensitive)
{
"propertyId": "email",
"op": "notLike",
"match": "%smith%"
}
All records without an email address containing smith (case sensitive).

startsWith
starts with
{
"propertyId": "email",
"op": "startsWith",
"match": "test"
}
All records with an email address starting with "test" (case sensitive).

endsWith
ends with
{
"propertyId": "email",
"op": "endsWith",
"match": "grouparoo.com"
}
All records with an email address ending with "@grouparoo.com" (case sensitive).

substring
includes the string
{
"propertyId": "email",
"op": "substring",
"match": "grouparoo"
}
All records with an email address containing "grouparoo".

iLike
is like (case insensitive)
{
"propertyId": "email",
"op": "like",
"match": "%smith%"
}
All records with an email address containing smith (case insensitive).

notILike
is not like (case insensitive)
{
"propertyId": "email",
"op": "notILike",
"match": "%smith%"
}
All records without an email address containing smith (case insensitive)).

The following operators are available on float properties:

exists
exists with any value
{
"propertyId": "ltv",
"op": "exists"
}
All records with an LTV value.

notExists
does not exist
{
"propertyId": "ltv",
"op": "notExists"
}
All records without an LTV value.

eq
is equal to
{
"propertyId": "ltv",
"op": "eq",
"match": "113932.97"
}
All records with an LTV of exactly $113,932.97.

ne
is not equal to
{
"propertyId": "ltv",
"op": "ne",
"match": "113932.97"
}
All records without an LTV of exactly $113,932.97.

gt
is greater than
{
"propertyId": "ltv",
"op": "gt",
"match": "113932.97"
}
All records with an LTV of greater than $113,932.97.

lt
is less than
{
"propertyId": "ltv",
"op": "lt",
"match": "113932.97"
}
All records with an LTV of less than $113,932.97.

gte
is greater than or equal to
{
"propertyId": "ltv",
"op": "gte",
"match": "113932.97"
}
All records with an LTV of exactly $113,932.97. or more

lte
is less than or equal to
{
"propertyId": "ltv",
"op": "lte",
"match": "113932.97"
}
All records with an LTV of exactly $113,932.97. or less

The following operators are available on integer properties:

exists
exists with any value
{
"propertyId": "visits",
"op": "notExists"
}
All records with any value in the "visits" field.

notExists
does not exist
{
"propertyId": "visits",
"op": "notExists"
}
All records with a null "visits" field.

eq
is equal to
{
"propertyId": "visits",
"op": "eq",
"match": "6"
}
All records with exactly six visits.

ne
is not equal to
{
"propertyId": "visits",
"op": "ne",
"match": "6"
}
All records without six visits.

gt
is greater than
{
"propertyId": "visits",
"op": "gt",
"match": "6"
}
All records with more than six visits.

lt
is less than
{
"propertyId": "visits",
"op": "ne",
"match": "6"
}
All records with less than six visits.

gte
is greater than or equal to
{
"propertyId": "visits",
"op": "gte",
"match": "6"
}
All records with six or more visits.

lte
is less than or equal to
{
"propertyId": "visits",
"op": "lte",
"match": "6"
}
All records with six or fewer visits.

The following operators are available on phoneNumber properties:

exists
exists with any value
{
"propertyId": "phone",
"op": "exists"
}
All records with a phone number listed.

notExists
does not exist
{
"propertyId": "phone",
"op": "notExists"
}
All records without a phone number listed.

eq
is equal to
{
"propertyId": "phone",
"op": "eq",
"match": "+1-555-555-5555"
}
All records with a phone number of +1-555-555-5555.

ne
is not equal to
{
"propertyId": "phone",
"op": "ne",
"match": "+1-555-555-5555"
}
All records without a phone number of +1-555-555-5555.

like
is like (case sensitive)
{
"propertyId": "phone",
"op": "like",
"match": "%780%"
}
All records with a phone number containing 780.

notLike
is not like (case sensitive)
{
"propertyId": "phone",
"op": "notLike",
"match": "%780%"
}
All records without a phone number containing 780.

startsWith
starts with
{
"propertyId": "phone",
"op": "startsWith",
"match": "+1"
}
All records with a phone number starting with +1.

endsWith
ends with
{
"propertyId": "phone",
"op": "startsWith",
"match": "80"
}
All records with a phone number ending with 80.

substring
includes the string
{
"propertyId": "phone",
"op": "startsWith",
"match": "80"
}
All records with a phone number containing the substring 80.

iLike
is like (case insensitive)
{
"propertyId": "phone",
"op": "notLike",
"match": "%780%"
}
All records without a phone number containing 780.

notILike
is not like (case insensitive)
{
"propertyId": "phone",
"op": "notLike",
"match": "%780%"
}
All records without a phone number containing 780.

The following operators are available on string properties:

exists
exists with any value
{
"propertyId": "lastName",
"op": "exists"
}
All records with a last name listed.

notExists
does not exist
{
"propertyId": "lastName",
"op": "notExists"
}
All records without a last name listed.

eq
is equal to
{
"propertyId": "lastName",
"op": "eq",
"match": "Ramirez"
}
All records with a last name of exactly Ramirez.

ne
is not equal to
{
"propertyId": "lastName",
"op": "ne",
"match": "Ramirez"
}
All records without a last name of exactly Ramirez.

like
is like (case sensitive)
{
"propertyId": "lastName",
"op": "like",
"match": "__m%"
}
All records with a last name that has a third letter of m.

notLike
is not like (case sensitive)
{
"propertyId": "lastName",
"op": "notLike",
"match": "__m%"
}
All records without a last name that has a third letter of m.

startsWith
starts with
{
"propertyId": "lastName",
"op": "startsWith",
"match": "Mc"
}
All records with a last name starting with Mc (case sensitive)

endsWith
ends with
{
"propertyId": "lastName",
"op": "endsWith",
"match": "son"
}
All records with a last name ending with son (case sensitive).

substring
includes the string
{
"propertyId": "lastName",
"op": "substring",
"match": "al"
}
All records with a last name containing al (case sensitive).

iLike
is like (case insensitive)
{
"propertyId": "lastName",
"op": "iLike",
"match": "%al%"
}
All records with a last name containing al (case insensitive).

notILike
is not like (case insensitive)
{
"propertyId": "lastName",
"op": "iLike",
"match": "%al%"
}
All records without a last name containing al (case insensitive).

The following operators are available on url properties:

exists
exists with any value
{
"propertyId": "company_url",
"op": "exists"
}
All records with a company URL listed.

notExists
does not exist
{
"propertyId": "company_url",
"op": "notExists"
}
All records without a company URL listed.

eq
is equal to
{
"propertyId": "company_url",
"op": "eq",
"match": "https://www.grouparoo.com"
}
All records with a company URL exactly equal to https://www.grouparoo.com.

ne
is not equal to
{
"propertyId": "company_url",
"op": "ne",
"match": "https://www.grouparoo.com"
}
All records without a company URL exactly equal to https://www.grouparoo.com.

like
is like (case sensitive)
{
"propertyId": "company_url",
"op": "like",
"match": "%grouparoo.com"
}
All records with a company URL containing anything and then grouparoo.com (case sensitive).

notLike
is not like (case sensitive)
{
"propertyId": "company_url",
"op": "like",
"match": "%grouparoo.com"
}
All records without a company URL containing anything and then grouparoo.com (case sensitive).

startsWith
starts with
{
"propertyId": "company_url",
"op": "startsWith",
"match": "https"
}
All records with a company URL starting with https (case sensitive).

endsWith
ends with
{
"propertyId": "company_url",
"op": "startsWith",
"match": ".com"
}
All records with a company URL ending with .com (case sensitive)

substring
includes the string
{
"propertyId": "company_url",
"op": "substring",
"match": "grouparoo"
}
All records with a company URL containing grouparoo (case sensitive)

iLike
is like (case insensitive)
{
"propertyId": "company_url",
"op": "substring",
"match": "%Grouparoo.com"
}
All records with a company URL containing anything followed by Grouparoo.com (case insensitive).

notILike
is not like (case insensitive)
{
"propertyId": "company_url",
"op": "substring",
"match": "%Grouparoo.com"
}
All records without a company URL containing anything followed by Grouparoo.com (case insensitive).

The following operators are available on boolean properties:

exists
exists with any value
{
"propertyId": "subscribed",
"op": "exists"
}
All records that do not have data about subscription status

notExists
does not exist
{
"propertyId": "subscribed",
"op": "notExists"
}
All records that have data about subscription status

eq
is equal to
{
"propertyId": "subscribed",
"op": "eq",
"match": "true"
}
All subscribers

ne
is not equal to
{
"propertyId": "subscribed",
"op": "ne",
"match": "true"
}
All non-subscribers

The following operators are available on date properties:

exists
exists with any value
{
"propertyId": "last_purchase_date",
"op": "exists"
}
All records who have made a purchase

notExists
does not exist
{
"propertyId": "last_purchase_date",
"op": "notExists"
}
All records who have not made a purchase

eq
is equal to
{
"propertyId": "last_purchase_date",
"op": "eq",
"match": "2021-05-25T04:45:00.000+00:00"
}
All records whose most recent purchase was on May 25, 2021 at exactly 4:45AM UTC.

ne
is not equal to
{
"propertyId": "last_purchase_date",
"op": "ne",
"match": "2021-05-25T04:45:00.000+00:00"
}
All records whose most recent purchase was anytime except May 25, 2021 at exactly 4:45AM UTC.

gt
is after
{
"propertyId": "last_purchase_date",
"op": "lt",
"match": "2020-12-25T04:45:00.00+00:00"
}
All records whose most recent purchase is after December 25, 2020 at 04:45:00 UTC.

lt
is before
{
"propertyId": "last_purchase_date",
"op": "lt",
"match": "2021-05-25T09:30:00.000+00:00"
}
All records whose most recent purchase is prior to May 25, 2021 9:30:00 AM UTC.

gte
is on or after
{
"propertyId": "last_purchase_date",
"op": "gte",
"match": "2019-06-14T18:16:00.000+00:00"
}
All records whose most recent purchase is exactly at or after June 14, 2019 at 6:16 PM UTC

lte
is on or before
{
"propertyId": "last_purchase_date",
"op": "lte",
"match": "2020-12-25T04:45:00.000+00:00"
}
All records whose most recent purchase is exactly at or before December 25, 2020 at 04:45:00 UTC.

relative_gt
is in the past
{
"propertyId": "last_purchase_date",
"op": "relative_gt",
"relativeMatchNumber": "60",
"relativeMatchUnit": "days"
}
All records whose most recent purchase is within the last 60 days

relative_lt
is in the future
{
"propertyId": "subscription_ends",
"op": "relative_lt",
"relativeMatchNumber": "30",
"relativeMatchUnit": "days"
}
All records whose subscription ends within the next 30 days

The following operators are available on email properties:

exists
exists with any value
{
"propertyId": "email",
"op": "exists"
}
All records with an email address listed.

notExists
does not exist
{
"propertyId": "email",
"op": "notExists"
}
All records without an email address listed.

eq
is equal to
{
"propertyId": "email",
"op": "eq",
"match": "hello@grouparoo.com"
}
All records with an email address exactly equal to hello@grouparoo.com.

ne
is not equal to
{
"propertyId": "email",
"op": "ne",
"match": "hello@grouparoo.com"
}
All records without an email address exactly equal to hello@grouparoo.com.

like
is like (case sensitive)
{
"propertyId": "email",
"op": "like",
"match": "%smith%"
}
All records with an email address containing smith (case sensitive).

notLike
is not like (case sensitive)
{
"propertyId": "email",
"op": "notLike",
"match": "%smith%"
}
All records without an email address containing smith (case sensitive).

startsWith
starts with
{
"propertyId": "email",
"op": "startsWith",
"match": "test"
}
All records with an email address starting with "test" (case sensitive).

endsWith
ends with
{
"propertyId": "email",
"op": "endsWith",
"match": "grouparoo.com"
}
All records with an email address ending with "@grouparoo.com" (case sensitive).

substring
includes the string
{
"propertyId": "email",
"op": "substring",
"match": "grouparoo"
}
All records with an email address containing "grouparoo".

The following operators are available on float properties:

exists
exists with any value
{
"propertyId": "ltv",
"op": "exists"
}
All records with an LTV value.

notExists
does not exist
{
"propertyId": "ltv",
"op": "notExists"
}
All records without an LTV value.

eq
is equal to
{
"propertyId": "ltv",
"op": "eq",
"match": "113932.97"
}
All records with an LTV of exactly $113,932.97.

ne
is not equal to
{
"propertyId": "ltv",
"op": "ne",
"match": "113932.97"
}
All records without an LTV of exactly $113,932.97.

gt
is greater than
{
"propertyId": "ltv",
"op": "gt",
"match": "113932.97"
}
All records without an LTV of greater than $113,932.97.

lt
is less than
{
"propertyId": "ltv",
"op": "lt",
"match": "113932.97"
}
All records without an LTV of less than $113,932.97.

gte
is greater than or equal to
{
"propertyId": "ltv",
"op": "gte",
"match": "113932.97"
}
All records without an LTV of exactly $113,932.97. or more

lte
is less than or equal to
{
"propertyId": "ltv",
"op": "lte",
"match": "113932.97"
}
All records without an LTV of exactly $113,932.97. or less

The following operators are available on integer properties:

exists
exists with any value
{
"propertyId": "visits",
"op": "notExists"
}
All records with any value in the "visits" field.

notExists
does not exist
{
"propertyId": "visits",
"op": "notExists"
}
All records with an empty "visits" field.

eq
is equal to
{
"propertyId": "visits",
"op": "eq",
"match": "6"
}
All records with exactly six visits.

ne
is not equal to
{
"propertyId": "visits",
"op": "ne",
"match": "6"
}
All records with any number of visits except six.

gt
is greater than
{
"propertyId": "visits",
"op": "gt",
"match": "6"
}
All records with more than six visits.

lt
is less than
{
"propertyId": "visits",
"op": "ne",
"match": "6"
}
All records with less than six visits.

gte
is greater than or equal to
{
"propertyId": "visits",
"op": "gte",
"match": "6"
}
All records with six or more visits.

lte
is less than or equal to
{
"propertyId": "visits",
"op": "lte",
"match": "6"
}
All records with 6 or fewer visits.

The following operators are available on phoneNumber properties:

exists
exists with any value
{
"propertyId": "phone",
"op": "exists"
}
All records with a phone number listed.

notExists
does not exist
{
"propertyId": "phone",
"op": "notExists"
}
All records without a phone number listed.

eq
is equal to
{
"propertyId": "phone",
"op": "eq",
"match": "+1-555-555-5555"
}
All records with a phone number of +1-555-555-5555.

ne
is not equal to
{
"propertyId": "phone",
"op": "ne",
"match": "+1-555-555-5555"
}
All records without a phone number of +1-555-555-5555.

like
is like (case sensitive)
{
"propertyId": "phone",
"op": "like",
"match": "%780%"
}
All records with a phone number containing 780.

notLike
is not like (case sensitive)
{
"propertyId": "phone",
"op": "notLike",
"match": "%780%"
}
All records without a phone number containing 780.

startsWith
starts with
{
"propertyId": "phone",
"op": "startsWith",
"match": "+1"
}
All records with a phone number starting with +1.

endsWith
ends with
{
"propertyId": "phone",
"op": "startsWith",
"match": "80"
}
All records with a phone number ending with 80.

substring
includes the string
{
"propertyId": "phone",
"op": "startsWith",
"match": "80"
}
All records with a phone number containing the substring 80.

The following operators are available on string properties:

exists
exists with any value
{
"propertyId": "lastName",
"op": "exists"
}
All records with a last name listed.

notExists
does not exist
{
"propertyId": "lastName",
"op": "notExists"
}
All records without a last name listed.

eq
is equal to
{
"propertyId": "lastName",
"op": "eq",
"match": "Ramirez"
}
All records with a last name of exactly Ramirez.

ne
is not equal to
{
"propertyId": "lastName",
"op": "ne",
"match": "Ramirez"
}
All records without a last name of exactly Ramirez.

like
is like (case sensitive)
{
"propertyId": "lastName",
"op": "like",
"match": "__m%"
}
All records with a last name that has a third letter of m.

notLike
is not like (case sensitive)
{
"propertyId": "lastName",
"op": "notLike",
"match": "__m%"
}
All records without a last name that has a third letter of m.

startsWith
starts with
{
"propertyId": "lastName",
"op": "startsWith",
"match": "Mc"
}
All records with a last name starting with Mc (case sensitive)

endsWith
ends with
{
"propertyId": "lastName",
"op": "endsWith",
"match": "son"
}
All records with a last name ending with son (case sensitive).

substring
includes the string
{
"propertyId": "lastName",
"op": "substring",
"match": "al"
}
All records with a last name containing al (case sensitive).

The following operators are available on url properties:

exists
exists with any value
{
"propertyId": "company_url",
"op": "exists"
}
All records with a company URL listed.

notExists
does not exist
{
"propertyId": "company_url",
"op": "notExists"
}
All records without a company URL listed.

eq
is equal to
{
"propertyId": "company_url",
"op": "eq",
"match": "https://www.grouparoo.com"
}
All records with a company URL exactly equal to https://www.grouparoo.com.

ne
is not equal to
{
"propertyId": "company_url",
"op": "ne",
"match": "https://www.grouparoo.com"
}
All records without a company URL exactly equal to https://www.grouparoo.com.

like
is like (case sensitive)
{
"propertyId": "company_url",
"op": "like",
"match": "%grouparoo.com"
}
All records with a company URL containing anything and then grouparoo.com (case sensitive).

notLike
is not like (case sensitive)
{
"propertyId": "company_url",
"op": "like",
"match": "%grouparoo.com"
}
All records without a company URL containing anything and then grouparoo.com (case sensitive).

startsWith
starts with
{
"propertyId": "company_url",
"op": "startsWith",
"match": "https"
}
All records with a company URL starting with https (case sensitive).

endsWith
ends with
{
"propertyId": "company_url",
"op": "startsWith",
"match": ".com"
}
All records with a company URL ending with .com (case sensitive)

substring
includes the string
{
"propertyId": "company_url",
"op": "substring",
"match": "grouparoo"
}
All records with a company URL containing grouparoo (case sensitive)

Destinations

Each Destination has a common set of attributess. Certain types of Destinations may have additional attributes, but all Destinations will have the common attributes. These common attributes, which may look very similar to other config object shapes in Grouparoo, are listed here:

  • id: (required) The unique value that defines this Destination.
  • name: (required) A display name for the Destination. Will be set to the id by default.
  • class: (required) Should be set to "Destination".
  • type: (required) The type of Destination will vary based on the app you're using. You likely don't want to change this value.
  • appId: (required) The id of the App (connection) that the Destination uses.
  • collection: (required) Which records should be exported from this Destination? Values can be "group", "model", or "none".
  • groupId: The id of the Group whose members should be exported to the Destination. (This is where Groups in Grouparoo really come in handy.) You need to choose collection: "group" to choose a groupId.
  • syncMode: (required): The SyncMode you wish to use for this Destination. Values can be "Sync", "Additive", and "Enrich".
  • mapping: See details below.
  • destinationGroupMemberships: See details below.

Example:

[
  {
    "id": "newsletter",
    "name": "newsletter",
    "class": "destination",
    "type": "mailchimp-export",
    "appId": "...",
    "collection": "...",
    "groupId": "...",
    "syncMode": "...",
    "options": {
      "listId": "...",
    },
    "mapping": {
      "email_address": "email",
      "FNAME": "firstName",
      "LNAME": "lastName",
    },
    "destinationGroupMemberships": {
      "High Value Customers!": "highValueCustomers",
    },
  }
]

Destination Mappings

Mappings are a series of key-value pairs that tell Grouparoo which Properties to export to the Destination for each Record. The structure may vary for each type of Destination, but generally, the keys are the fields in the Destination, while the values are the id values for the Properties in Grouparoo.

Here's an example of a Mailchimp mapping configuration:

{
  "mapping": {
    "email": "email",
    "FNAME": "firstName",
    "LNAME": "lastName"
  }
}

Notice the odd structure of the keys. That's specific to Mailchimp.

Destination Group Membership

Grouparoo also provides the ability to attach Group membership to the exported data. How this looks within the Destination is specific to the Destination. But the shape of the config file largely remains the same. In most Destinations, Destination Group Memberships become either tags (Mailchimp) or lists (HubSpot), or are appended as additional properties to the Record (Customer.io).

destinationGroupMemberships are a set of key-value pairs that tell Grouparoo which memberships to publish to the Destination. For example, you may have sent all your users to a Mailchimp Destination, but you wanted to tag high-value customers.

In the example below, the key represents the tag in Mailchimp, while the value is the id of the Group in Grouparoo:

{
  "destinationGroupMemberships": {
    "High Value Customers!": "highValueCustomers"
  }
}

Validating & Applying Your Config

You can validate your config at any time using the validate command:

$ grouparoo validate

And you can apply that config (save it to your Grouparoo application's database) using the apply command:

$ grouparoo apply