Providing Transaction Bandwidth

Cascade Bandwidth Marketplace

Solving Transaction Delivery Problems

Put a new server on the Internet, and it will be attacked within minutes. Make that server a validator on a blockchain, and robots will abuse it with a profit motive. Given enough abuse, validators will become overloaded, and the blockchain will become unusable. One of the biggest challenges in the blockchain space is managing validator network traffic for a better user experience.

The Solana blockchain improves the user experience by providing multiple pathways for transaction (“TX”) delivery. More delivery options make the whole system more robust. Solana TX routes include public ports on validators, Jito Labs’ auctions, and the inelegantly named Stake-Weighted Quality of Service (“SWQoS”) to deliver transactions through staked validators.

Fees and features vary by route. The public ports are free but subject to network congestion. Jito bidders pay extra fees, AKA “tips,” to win auctions and write bundles of TXs at the beginning of every block. With SWQoS, Validators offer reserved TX bandwidth for a fee so that users can bypass congestion on the public ports.

Blockchain RPC providers and their users would like to reserve bandwidth to deliver TXs faster. Blockchain validators would like to sell bandwidth up to a limit determined by their stake weight. The Cascade Marketplace brings buyers and sellers together to set a fair clearing price through regular auctions. After the auction, bandwidth is delivered through Cascade’s network of affiliated validators.

TX Bandwidth

Every validator gets awarded Transaction Bandwidth at the start of each epoch. The total bandwidth for all stake is 250 000 PPS. You can calculate your Transaction Bandwidth by multiplying your percentage of the total stake at any given time with the total Transaction Bandwidth available. If you have 0.1% of total stake, the amount of Transaction Bandwidth per second available to you is 0.1% * 250 000 = 250 PPS. Transaction Bandwidth is measured in packets per second (“PPS”), where one packet represents one transaction or a transaction retry.

Triton One as the Marketplace Operator

Triton One, the marketplace Operator, will collect all bids and provide an updated, anonymized order book for all participants to see. Bidding is open until the end of the current epoch. After the epoch ends, the Operator will announce the winners and the Floor Price.

The Operator will then arrange for the reserved bandwidth to be delivered through the Cascade network. Bids on the marketplace are done in USD. At the end of the month the buyer receives an invoice which can be paid in USD.

Cascade Network Operation

Auction winners will receive a private endpoint on the Triton One Cascade Network for transaction delivery. The current version acts as a relay-only service without retries or simulations. Simulations will be added soon:tm: Transactions will be relayed through staked validators within the selected region.

Excess packets above the bandwidth limit will be dropped. There are no credits for unused bandwidth or carry-overs to the next period—use it or lose it.

Revenue Sharing

The revenue from the Cascade Marketplace is measured and paid out in USD. It is split 45% to Triton One and 55% for the bandwidth provider and it's delegators.

AML Compliance

The Cascade Marketplace and network activity are all off-chain. Triton One and other participants must comply with anti-money laundering (AML) laws. Therefore, Cascade Marketplace will require contracts and identity verification (KYB/KYC) when money changes hands (Buyers, Resellers, Validators, and some Stakeholders). This is a B2B marketplace, so AML compliance should be familiar to all participants.

Getting started providing bandwidth to Cascade

Install Yellowstone Jet

You can build Jet from https://github.com/rpcpool/yellowstone-jet.

The binary you want is the Jet binary. This is our transaction forwarder. It has multiple interfaces, including a normal Solana RPC interface and a gRPC interface that allows you to connect it to receive transactions from the Cascade marketplace.

You can install Yellowstone Jet directly on your validator node, or you can run it on a different node separate from your validator. The only requirement is to load your validator identity into Jet so that Jet can assume that identity and send transactions from it.

Receive a Marketplace Endpoint

We will provide you with a marketplace access token and endpoint. Connect your Jet instances to this endpoint.

Your Triton One contact will set up with a marketplace endpoint, including;

• RPC endpoint

• gRPC endpoint

• gRPC x-token

• Jet Gateway endpoint

Configure Jet

Create the following configuration file using details provided by Triton. This configuration will allocate all your bandwidth to Cascade and report bandwidth to the Cascade Marketplace.

tracing:
  json: true # change to `true` for production

identity:
  # Do not send transactions if Quic identity doesn't match specified one
  expected: << validator identity pubkey >> 
  # Keypair, if you don't want to use dynamic loading
  #keypair: << validator identity key >>

# RPC & gRPC for upstream validator
upstream:
  # gRPC service endpoint
  primary_grpc:
    endpoint: <<either grpc running locally on your validator or the rpcpool.com endpoint you have been provided>>
    # Optional token for access to gRPC
    x_token: << either remove this if you use your own validator grpc or use the token provided by triton >>
  secondary_grpc: null # leave null if using triton grpc, if using your own grpc put the triton one here

  # RPC endpoint
  rpc: << rpc endpoint https://...rpcpol.com/<jet agteway token> >>
  # Cluster nodes information update interval in milliseconds
  cluster_nodes_update_interval: 30s

# Jet gateway
jet_gateway:
  endpoints:
    - << jet gateway endpoint https://...rpcpool.com >>
  x_token: << jet gateway token aaa-aaa-ddd-eee >>

# Admin server listen options
listen_admin:
  # RPC listen address
  bind:
    - 127.0.0.1:11888 # if you want to track data with prometheus specify a different address here
  # Number of RPC threads to process requests
  # worker_threads: 2

# Solana-like server listen options
listen_solana_like:
  # Solana like RPC listen address
  bind:
    - 127.0.0.1:11899
  # Number of RPC threads to process requests
  #worker_threads: 2
  # Allow to do sanitize check on RPC server (required for ALTs), supported only on patched nodes
  # If option set to `true`` then Jet would check `sanitizeTransaction` method before start
  # See https://github.com/rpcpool/solana-public/tree/v1.17.31-rpc-sanitize-tx
  proxy_sanitize_check: true
  # Allow to do preflight check on RPC server (simulateTransaction)
  proxy_preflight_check: true

# Send retry options
send_transaction_service:
  # Default max retries of sending transaction
  default_max_retries: 0
  # Service max retries
  service_max_retries: 0
  # Stop send transaction when landed at specified commitment
  stop_send_on_commitment: confirmed
  # Send transaction to number of leaders
  leader_forward_count: 4
  # Try to send transaction every retry_rate duration
  retry_rate: 1s
  # Drop transactions from the pool once max retries limit is reached (landed statistic would be invalid)
  # relay_only_mode: false

# Quic config
quic:
  # Total number of pools (one pool per remote address)
  # Solana value is 1024.
  # https://github.com/solana-labs/solana/blob/v1.17.31/connection-cache/src/connection_cache.rs#L22
  connection_max_pools: 256
  # TPU connection pool size per remote address
  # Default is `solana_tpu_client::tpu_client::DEFAULT_TPU_CONNECTION_POOL_SIZE` -- 4
  connection_pool_size: 1
  # Number of immediate retries in case of failed send
  send_retry_count: 1
  # Kind of Quic port: `normal` or `forwards`
  tpu_port: forwards
  # Quic handshake timeout.
  # Default is `solana_sdk::quic::QUIC_CONNECTION_HANDSHAKE_TIMEOUT` -- 60s
  connection_handshake_timeout: 4s
  # Maximum duration of inactivity to accept before timing out the connection.
  # https://docs.rs/quinn/0.10.2/quinn/struct.TransportConfig.html#method.max_idle_timeout
  # Default is `solana_sdk::quic::QUIC_KEEP_ALIVE` -- 2s
  max_idle_timeout: 2s
  # Period of inactivity before sending a keep-alive packet
  # https://docs.rs/quinn/0.10.2/quinn/struct.TransportConfig.html#method.keep_alive_interval
  # Default is `solana_sdk::quic::QUIC_KEEP_ALIVE` -- 1s
  keep_alive_interval: 1s
  # Send tx timeout, for batches value multipled by number of transactions in the batch
  # Solana default value is 10 seconds
  send_timeout: 10s
  # The outbound udp port range to use
  endpoint_port_range:
    start: 35000
    end: 45000
  # See `solana_streamer::nonblocking::quic::compute_max_allowed_uni_streams`
  # Minumum value is `QUIC_MAX_UNSTAKED_CONCURRENT_STREAMS` -- for unstaked nodes, 128
  # Value for staked calculated from total stake, but maximum is `QUIC_MAX_STAKED_CONCURRENT_STREAMS`
  send_max_concurrent_streams: 128

If you want to restrict how much bandwidth you provide to cascade, provide the following parameter:

Set up a jet systemd service

Once you have placed the configuration file above in a location such as /etc/yellowstone-jet.yml you can create a systemd service in /etc/systemd/system/.

[Unit]
Description=Yellowstone Jet transaction forwarder
After=network-online.target
StartLimitInterval=0
StartLimitIntervalSec=0

[Service]
Type=simple
User=yellowstone-jet
Group=yellowstone-jet
PermissionsStartOnly=true
ExecStart=/usr/local/bin/yellowstone-jet --config /etc/yellowstone-jet.yml

Environment=RUST_LOG="warn"

SyslogIdentifier=yellowstone-jet
KillMode=process
Restart=always
RestartSec=5

LimitNOFILE=700000
LimitNPROC=700000

LockPersonality=true
NoNewPrivileges=true
PrivateTmp=true
ProtectHome=true
RemoveIPC=true
RestrictSUIDSGID=true

ProtectSystem=full

[Install]
WantedBy=multi-user.target

Loading your validator identity into Jet

You can choose to load your validator identity into Jet statically or dynamically. This depends on your setup.

To load the key statically, provide a file path or the key itself as the keypair argument to the `identity` flag.

If you want to load your key statically, you must provide the public key in the `expected` flag and then use the admin RPC interface to load the key at runtime (after Jet starts). The admin RPC interface exposes setIdentity/getIdentity JSON-RPC methods you can employ.

Checking metrics to see that it is working

Jet publishes metrics under its admin RPC, access these to see that your Jet Gateway instance is working:

curl -s http://127.0.0.1:11888/metrics

Firewall ports and other config

Jet does not open any external listening ports and relies on an outbound connection to our Jet Gateway to receive transactions.