src.core.process.extract module

the file, which mainly manages the data extraction process

class src.core.process.extract.ImportPipeline(*values)[source]

Bases: Enum

possible pipelines, we use currently

CLIENT

the pipeline for the client dumped data

Type:

ClientKeys

MATCHV5

the pipeline for the matchv5 gathered data

Type:

MatchV5Keys

CLIENT = <enum 'ClientKeys'>
MATCHV5 = <enum 'MatchV5Keys'>
src.core.process.extract.classify(member)[source]

classifys the GameTabletype for a Key

Parameters:

member (ClientKeys | matchV5Keys) – the member of one of the enums

Return type:

the table, which it belongs to

src.core.process.extract.extractRawTables(data, pipe, typ)[source]

Collects all member of a enum class (see head of this file) and executes these onto a data dict, to extract the data and format it into three outcomes: PLAYER-, META- and TEAMDATA

Parameters:
Returns:

dataframes – contains the three major tables: metadata, playerdata, teamdata

Return type:

list[pd.DataFrame]

src.core.process.extract.getData(data, pathKey, typ, metaKey=None, agg=False)[source]

parses the data according the passed pathKey

Parameters:

  • data (dict) – the raw data, extracted from a .json gamefile

  • pathKey (MatchV5Keys | ClientKeys) – the path to the data in the json tree of the gamefile

  • metaKey (list[list[str]] | None, optional) – the path to a meta variable. this is only used, if the jsontree contains deeper lists, that we have to parse in an extra step.

  • agg (bool, optional) – if passed, the tables gets aggregated with indexbyoneVariable

  • typ (ImportType)

Returns:

dataframe – the resulting table

Return type:

pd.DataFrame

src.core.process.extract.pipeToAgg: dict[ImportPipeline, dict] = {ImportPipeline.CLIENT: {ClientKeys.TEAM_2: ['teams', 'teamId']}, ImportPipeline.MATCHV5: {MatchV5Keys.FRAME: ['info', 'matchId'], MatchV5Keys.PLAYER_3: ['info', 'participants', 'participantId'], MatchV5Keys.TEAM_2: ['info', 'teams', 'teamId']}}

dict, which maps the pipeline to the aggregation helper

src.core.process.extract.pipeToMeta: dict[ImportPipeline, dict] = {ImportPipeline.CLIENT: {ClientKeys.PLAYER_1: 'gameId', ClientKeys.TEAM_1: 'gameId'}, ImportPipeline.MATCHV5: {MatchV5Keys.EVENTS: ['info', 'matchId'], MatchV5Keys.PLAYER_1: ['info', 'gameId'], MatchV5Keys.TEAM_1: ['info', 'gameId']}}

dict, which maps the pipeline to the metapath helper

Dependency Diagrams (without externals):

digraph imports { rankdir=LR; node [shape=box]; "src.core.process.reading" -> "src.core.process.extract"; }

Import dependencies (collapsed)

Dependency Diagrams:

digraph imports { rankdir=LR; node [shape=box]; "src.core.process.reading" -> "src.core.process.extract"; }

Import dependencies (collapsed)