EVM Manifest File
EVM Manifest File
The Manifest project.ts file can be seen as an entry point of your project and it defines most of the details on how SubQuery will index and transform the chain data. This is the specific details for Ethereum and other EVM-compatible chains, please refer to the top level manifest documentation for more general details.
Below is a standard example of a basic EVM project.ts.
import {
EthereumProject,
EthereumDatasourceKind,
EthereumHandlerKind,
} from "@subql/types-ethereum";
// Can expand the Datasource processor types via the generic param
const project: EthereumProject = {
specVersion: "1.0.0",
version: "0.0.1",
name: "ethereum-subql-starter",
description:
"This project can be use as a starting point for developing your new Ethereum SubQuery project",
runner: {
node: {
name: "@subql/node-ethereum",
version: ">=3.0.0",
},
query: {
name: "@subql/query",
version: "*",
},
},
schema: {
file: "./schema.graphql",
},
network: {
/**
* chainId is the EVM Chain ID, for Ethereum this is 1
* https://chainlist.org/chain/1
*/
chainId: "1",
/**
* These endpoint(s) should be non-pruned archive nodes
* Public nodes may be rate limited, which can affect indexing speed
* When developing your project we suggest getting a private API key
# We suggest providing an array of endpoints for increased speed and reliability
*/
endpoint: ["https://ethereum.rpc.subquery.network/public"],
},
dataSources: [
{
kind: EthereumDatasourceKind.Runtime,
startBlock: 4719568,
options: {
// Must be a key of assets
abi: "erc20",
// # this is the contract address for wrapped ether https://etherscan.io/address/0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2
address: "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2",
},
assets: new Map([["erc20", { file: "./abis/erc20.abi.json" }]]),
mapping: {
file: "./dist/index.js",
handlers: [
{
kind: EthereumHandlerKind.Call,
handler: "handleTransaction",
filter: {
/**
* The function can either be the function fragment or signature
* function: '0x095ea7b3'
* function: '0x7ff36ab500000000000000000000000000000000000000000000000000000000'
* function: null - this will filter for native transfers that have no contract calls
*/
function: "approve(address spender, uint256 rawAmount)",
},
},
{
kind: EthereumHandlerKind.Event,
handler: "handleLog",
filter: {
/**
* Follows standard log filters https://docs.ethers.io/v5/concepts/events/
* address: "0x60781C2586D68229fde47564546784ab3fACA982"
*/
topics: [
"Transfer(address indexed from, address indexed to, uint256 amount)",
],
},
},
],
},
},
],
repository: "https://github.com/subquery/ethereum-subql-starter",
};
// Must set default to the project instance
export default project;Below is a standard example of the legacy YAML version (project.yaml).
Legacy YAML Manifest
specVersion: "1.0.0"
name: "ethereum-subql-starter"
version: "0.0.1"
runner:
node:
name: "@subql/node-ethereum"
version: "*"
query:
name: "@subql/query"
version: "*"
description: "This project can be use as a starting point for developing your new Ethereum SubQuery project"
repository: "https://github.com/subquery/ethereum-subql-starter"
schema:
file: "./schema.graphql"
network:
# chainId is the EVM Chain ID, for Ethereum this is 1
# https://chainlist.org/chain/1
chainId: "1"
# This endpoint must be a public non-pruned archive node
# We recommend providing more than one endpoint for improved reliability, performance, and uptime
# Public nodes may be rate limited, which can affect indexing speed
# When developing your project we suggest getting a private API key and using endpoint config
endpoint: ["https://ethereum.rpc.subquery.network/public"]
dataSources:
- kind: ethereum/Runtime
startBlock: 15695385
options:
# Must be a key of assets
abi: erc20
address: "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2" # this is the contract address for wrapped ether https://etherscan.io/address/0xc02aaa39b223fe8d0a0e5c4f27ead9083c756cc2
assets:
erc20:
file: "erc20.abi.json"
mapping:
file: "./dist/index.js"
handlers:
- handler: handleTransaction
kind: ethereum/TransactionHandler
filter:
## The function can either be the function fragment or signature
# function: '0x095ea7b3'
# function: '0x7ff36ab500000000000000000000000000000000000000000000000000000000'
# function: null - this will filter for native transfers that have no contract calls
function: approve(address spender, uint256 rawAmount)
- handler: handleLog
kind: ethereum/LogHandler
filter:
topics:
## Follows standard log filters https://docs.ethers.io/v5/concepts/events/
- Transfer(address indexed from, address indexed to, uint256 amount)
# address: "0x60781C2586D68229fde47564546784ab3fACA982"Overview
Datasource Spec
Defines the data that will be filtered and extracted and the location of the mapping function handler for the data transformation to be applied. EVM datasources also contain references to contract ABIs that are used for filtering and decoding data.
| Field | Type | Description |
|---|---|---|
| kind | string | ethereum/Runtime |
| startBlock | Integer | This changes your indexing start block for this datasource, set this as high as possible to skip initial blocks with no relevant data |
| endBlock | Integer | This sets a end block for processing on the datasource. After this block is processed, this datasource will no longer index your data. Useful when your contracts change at a certain block height, or when you want to insert data at genesis. For example, setting both the startBlock and endBlock to 320, will mean this datasource only operates on block 320 |
| mapping | Mapping Spec | |
| options | { abi: string; address: string; } | The name of the abi in assets to use for decoding data and an address for the contract to filter |
| assets | Map<string, { file: string }> | A map of contract names to abi file paths |
Data Sources and Mapping
In this section, we will talk about the default Ethereum runtime and its mapping. Here is an example:
{
dataSources: [
{
kind: EthereumDatasourceKind.Runtime, // Indicates that this is default runtime
startBlock: 1, // This changes your indexing start block, set this higher to skip initial blocks with less data
options: {
// Must be a Record of assets
abi: "erc20",
// # this is the contract address for your target contract
address: "0xC02aaA39b223FE8D0A0e5C4F27eAD9083C756Cc2",
},
assets: new Map([["erc20", { file: "./abis/erc20.abi.json" }]]),
mapping: {
file: "./dist/index.js", // Entry path for this mapping
handlers: [
/* Enter handers here */
],
},
},
];
}Mapping Handlers and Filters
The following table explains filters supported by different handlers.
Your SubQuery project will be much more efficient when you only use TransactionHandler or LogHandler handlers with appropriate mapping filters (e.g. NOT a BlockHandler).
| Handler | Supported filter |
|---|---|
| ethereum/BlockHandler | modulo, timestamp |
| ethereum/TransactionHandler | function filters (either be the function fragment or signature), from (address), to (address), type ("0x0" for legacy, "0x1" for access type lists, "0x2" for dynamic fees and "0x3" for blob transactions) |
| ethereum/LogHandler | topics filters, and address |
Default runtime mapping filters are an extremely useful feature to decide what block, event, or transaction will trigger a mapping handler.
Only incoming data that satisfies the filter conditions will be processed by the mapping functions. Mapping filters are optional but are highly recommended as they significantly reduce the amount of data processed by your SubQuery project and will improve indexing performance.
modulo and timestamp are common block filters and are described in the overview
Real-time indexing (Block Confirmations)
As indexers are an additional layer in your data processing pipeline, they can introduce a massive delay between when an on-chain event occurs and when the data is processed and able to be queried from the indexer.
SubQuery provides real time indexing of unconfirmed data directly from the RPC endpoint that solves this problem. SubQuery takes the most probabilistic data before it is confirmed to provide to the app. In the unlikely event that the data isn’t confirmed and a reorg occurs, SubQuery will automatically roll back and correct its mistakes quickly and efficiently - resulting in an insanely quick user experience for your customers.
To control this feature, please adjust the --block-confirmations command to fine tune your project and also ensure that historic indexing is enabled (enabled by default). The default block confirmations for SubQuery projects is currently 200 blocks.
