Resource Management¶

Each subscription comes with a specific amount of cloud resources for data storage and computation. The following resources are recorded:

• Compute Credits: unit of measure to quantify resource consumption when running an analysis tool
• Storage: size of all files in active storage (i.e., immediately available for running analyses and downloading)
• Archived Storage: size of all files archived
• Downloaded Files: size of all files downloaded from IDEAS

Use the following sections to learn about the following topics:

• Compute credits
• Viewing resource consumption
• User-specified computing resources
  How to customize what resources are used when running analyses.
• Archiving data
  How to archive data to manage resources.

Compute credits¶

Compute credits are a unit of measure to quantify resource consumption when running analysis tools in IDEAS. These credits take into account the total RAM, memory, and GPUs required in addition to vCPUs. Refer to the following table to note how many compute credits are used for varying computing resources. Analyses requiring more computational resources to perform efficiently will use more compute credits.

Refer to the following section Viewing Resource Consumption for where to find how many compute credits you've used. Note that each tool described in the Analysis Tools section lists how many compute credits are used by the tool.

Viewing resource consumption¶

There are several locations in IDEAS where you can view your resource consumption and/or consumption rates for analysis tools:

Organization home page¶

From your user home page (left side panel), click your organization link to view the resource consumption table (displayed in the left side panel). Image below shows the left side panels for the user home page (left) and organization home page (right):

Analysis table¶

The analysis table lists the following consumption resource information:

• Total usage for all jobs completed

  In the example below, the jobs that were completed consumed 0.05 compute credits.

Analysis tools documentation¶

At the top of each tool description in the IDEAS documentation you can find the default compute credits consumed per hour as shown for the preprocessing tool below.

User-specified computing resources¶

When building an analysis table, you may configure the computing resources you want to allocate for the analysis task to run. For example, the image below shows that the end-to-end CNMFe task has been selected to use the following resources: 8 vCPUs, 64 GB RAM, 2 credits/hr. To limit the consumption of your resources, you may choose an option using fewer credits. Note that this may cause the analysis to run slower.

To help decide whether the resources allocated should be modified, please refer to the logs from previously executed tasks. The Resource Usage tab lists the CPU and RAM consumed and can help you appropriately select resources for subsequent tasks.

Archive data¶

Archive storage offers a cost-effective solution for long-term retention of important data. Moving infrequently accessed files to archive storage frees up space in your active storage environment. Note that analyses cannot be performed on archived files unless they are unarchived.

Individual files can be archived by clicking either the menu for the file (image below, left) or by clicking on the Archive icon (snowflake) in the preview panel (image below, right).