Learn how your usage is calculated
In many cases, it is helpful to know how much usage you will incur before kicking off a large modeling run, or even when you are just experimenting.
Purchase of the DL Platform is through a contract, and the usage discussed here is measured in units and billed on a per-unit basis. Contracts may include a number of prepaid units, which are referred to here as an entitlement.
Processing Units
A Processing Unit is 1,000 tiles, where a tile is 512x512 pixels, with one band, and one timestamp.
Multipliers are applied for:
- Number of images
- Number of bands
- Size of AOI
To understand how many Processing Units a call will use, apply this formula:
tiles = ceiling(x_pixels/512) * ceiling(y_pixels/512)
processing_units = num_images * num_bands * tiles / 1000
It's important to note that the Processing Unit calculation includes alpha bands. If you are working with Catalog Products that have an alpha band, calls that don't specify mask_alpha=False will automatically include the alpha band. This information can be found in the documentation for the Scene and SceneCollection rasterization methods:
mask_alpha (bool or str or None, default None) – Whether to mask pixels in all bands where the alpha band of all scenes is 0. Provide a string to use an alternate band name for masking. If the alpha band is available for all scenes in the collection and mask_alpha is None, mask_alpha is set to True. If not, mask_alpha is set to False.
Processing Unit entitlements are honored by comparing contract usage to-date with the entitlement amount. For usage beyond the entitlement, Processing Units are metered. Each metering period, whole-number Processing Unit usage will be metered, with partial usage carried over to subsequent metering periods.
Example 1: Stacking Imagery
For example, if you are pulling a stack of 10 images with RGB and Near-Infrared bands from a Catalog product with an alpha band (and you do not specify mask_alpha=False), and your AOI is 1024x1024 pixels:
tiles = ceiling(1024/512) * ceiling(1024/512) = 4
processing_units = 10 * 5 * 4 / 1000 = 0.2
If you performed the same operation but for 1,000 AOIs instead of just one, you would use 2,000 Processing Units.
Example 2: Many Small Fields
AOIs smaller than 512x512 pixels will not reduce the number of Processing Units consumed. For instance, if you have 5,000 agricultural fields (AOIs), each 10-30 pixels on each side, and want to pull 12 bands from the most recent image for each of those:
tiles = ceiling(30/512) * ceiling(30/512) = 1
per_field_pu = 1 * 12 * 1 / 1000 = 0.012
total_pu = 5000 * 0.012 = 60
If you performed this operation every week, you could expect to consume 60 Processing Units weekly for a total of 3,120 Processing Units for the year.
Sample Python Calculation
import math
def get_pu_used(num_scenes, num_bands, arr_shape):
"""
Calculate the Processing Units consumed by a scenes call
Parameters
----------
num_scenes : int
The number of scenes in the request. Typically
`len(scenes)`
num_bands : int
The number of bands in the request, including alpha or
other mask band
arr_shape : tuple of array shape (x, y)
The x and y shape of the returned array in pixels.
Typically array.shape[-2:] when the spatial dimensions
are the last two (the default)
Returns
-------
processing_units : float
"""
pixels_x, pixels_y = arr_shape
blocks_x = math.ceil(pixels_x/512)
blocks_y = math.ceil(pixels_y/512)
blocks_total = blocks_x*blocks_y
processing_units = num_scenes * num_bands * blocks_total / 1000
return processing_units
get_pu_used(10, 5, (1024,1024))
# 0.2
5000 * get_pu_used(1, 12, (30,10))
# 60
Catalog Storage
Catalog storage is consumed by storing data in the Descartes Labs Catalog. The total storage used is calculated by adding up the file sizes for files stored in the Catalog.
Catalog Storage entitlements are honored by comparing usage each metering period with the entitlement amount. For storage beyond the entitlement, Catalog Storage is metered daily by the GiB-day. Each metering period, usage less than one GiB-day will be rounded up to 1 GiB-day. For each subsequent GiB-day, usage will be rounded to the nearest integer.
Example
If your contract specifies an entitlement of 4 TiB and you store 5 TiB of data in the Catalog every day for a month, you would be metered for 1 TiB. In terms of GiB-days, this would amount to 1,024 GB-days x 30 days, or 30,720 GB-days.
Compute Units
For contracts specifying a Compute Unit entitlement. Compute Units are consumed through the use of Descartes Labs compute services. A Compute Unit is measured as the number of CPU-hours consumed. Factors such as GPU usage, memory requested, and additional instance configuration may affect the number of Compute Units used.
Viewing Your Usage
You can view usage associated with your account at https://iam.descarteslabs.com/
Other Usage Calculation & Metering Info
- Unless otherwise noted, Processing Unit usage is metered every hour and Catalog Storage usage is metered every day.
- Unless otherwise noted, Processing Unit usage is aggregated on a per-user and per-hour basis prior to being metered.
-
Usage within your entitlement can only be viewed at https://iam.descarteslabs.com/. This interface also allows for viewing detailed real-time account-specific usage.