plexus.data package

The Plexus data packages are for loading and caching data for use in training and evaluating scores. The abstract base class DataCache defines the interface for the various data cache implementations. Each should override the parameters to define whatever parameters it needs, and then implement the load_data method to load a dataframe from the cache.

Submodules

• plexus.data.AWSDataLakeCache module
  • AWSDataLakeCache
    • AWSDataLakeCache.Parameters
      • AWSDataLakeCache.Parameters.local_cache_directory
      • AWSDataLakeCache.Parameters.model_config
    • AWSDataLakeCache.__init__()
    • AWSDataLakeCache.download_content_item()
    • AWSDataLakeCache.execute_athena_query()
    • AWSDataLakeCache.execute_batch_athena_queries()
    • AWSDataLakeCache.get_query_results()
    • AWSDataLakeCache.load_dataframe()
    • AWSDataLakeCache.process_content_item()
    • AWSDataLakeCache.split_into_batches()
• plexus.data.DataCache module
  • DataCache
    • DataCache.Parameters
      • DataCache.Parameters.class_name
      • DataCache.Parameters.model_config
    • DataCache.__init__()
    • DataCache.load_dataframe()
    • DataCache.log_validation_errors()
• plexus.data.HuggingFaceDataCache module
  • HuggingFaceDataCache
    • HuggingFaceDataCache.Parameters
      • HuggingFaceDataCache.Parameters.model_config
      • HuggingFaceDataCache.Parameters.name
    • HuggingFaceDataCache.__init__()
    • HuggingFaceDataCache.analyze_dataset()
    • HuggingFaceDataCache.load_dataframe()
    • HuggingFaceDataCache.verify_dataset()