wtgseal package¶

Submodules¶

wtgseal.cli module¶

wtgseal.maker module¶

Functions to generate locust code.

This module contains functions to generate locust code, i.e. it defines tasks, tasksets and users.

wtgseal.maker.cmddef_to_str(cmddef: Tuple[int, str], indentby: str = ' ') → str[source]¶: Convert a command definition into a string.

wtgseal.maker.setup_blank_line(n: int = 1) → List[Tuple[int, str]][source]¶: Generate representation for a given number of blank lines.

wtgseal.maker.setup_csv_stats_interval(t: int) → List[Tuple[int, str]][source]¶

Generate import and settings for CSV stats interval.

Generate code to import locust.stats module and set the property CSV_STATS_INTERVAL_SEC.

Parameters:	t ({int}) – How frequently (in seconds) CSV data are to be written. This has only effect when –csv-full-history is passed to locust run.
Returns:	A list of code representation, where each item represents a line of import.
Return type:	BlockDef

wtgseal.maker.setup_header(*, dist: str = 'wtg-seal', version: str = 'unknown', url: str = 'https://github.com/mchoji/wtg-seal') → List[Tuple[int, str]][source]¶

Generate a simple header with dist, version and url information.

Generate a program header citing the distribution name from where it was generated, the current version and the url where one can find further information.

wtgseal.maker.setup_import() → List[Tuple[int, str]][source]¶

Generate the import lines for a locust file.

Generate code to import the modules needed for running a locust file.

Returns:	A list of code representation, where each item represents a line of import.
Return type:	BlockDef

wtgseal.maker.setup_task(name: str = 'task0', uri: List[str] = ['/'], *, weight: int = 1, group_name: str = None, indlevel: int = 0) → List[Tuple[int, str]][source]¶

Generate code to define a locust task.

Generate code to define a locust task according to the given parameters.

Parameters:	name ({str}, optional) – The name for the task to be generated (the default is ‘task0’) weight ({int}, optional) – The weight for the generated task (the default is 1) uri ({List[str] = ["/", ]}) – A list of URIs, each starting with a backslash like “/index.html” group_name (str) – Group URLs together in Locust’s statistics using this name (the default is None, which leads to calculate statistics for each URL separately) indlevel ({int}, optional) – The indentation level where the task definition should begin (the default is 0, which leads to code beginning at the left margin)
Returns:	A list where each item represents a line of code to be generated. Each Tuple[int, str] consists of the indentation level for the code and the code itself. None is returned in case uri is not an iterable.
Return type:	List[Tuple[int, str]]

wtgseal.maker.setup_taskset(name: str = 'MyTaskSet') → List[Tuple[int, str]][source]¶: Generate representation of TaskSet subclass definition.

wtgseal.maker.setup_user(name: str = 'MyUser', taskset: str = 'MyTaskSet', *, wait_seed: int = 1, weight: int = 1, indlevel: int = 0) → List[Tuple[int, str]][source]¶

Generate a model for user behaviour.

Generate a representation for a User subclass, which represents an user behaviour. The representation contains the class name, taskset, and a wait time based on a Pareto distribution.

Parameters:	name ({str}, optional) – The new class name (the default is ‘MyUser’). taskset ({str}, optional) – The tasksets this class will use (the default is ‘MyTaskSet’). wait_seed ({int}, optional) – The seed to be used for the Pareto distribution from where the wait times are retrieved (the default is 1). weight ({int}, optional) – The weight for this class. The greater this value, the greater the chances this class will be spawned. Only important in case you have more than one User subclass and with they be spawned at different rates. indlevel ({int}, optional) – The indentation level where the task definition should begin (the default is 0, which leads to code beginning at the left margin).
Returns:	A list where each item represents a line of code to be generated. Each Tuple[int, str] consists of the indentation level for the code and the code itself.
Return type:	List[Tuple[int, str]]

Notes

An OFF time is defined in this class by the wait_time attribute. This attribute defines how long a locust will wait after each task. Here, we define it as a pareto distribution with shape 1.4 and scale 1.0, based on SURGE implementation [1]_.

References

[1]	Barford, P., & Crovella, M. (1998, June). Generating representative web workloads for network and server performance evaluation. In Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems (pp. 151-160).

wtgseal.utils module¶

Utilities for wtg-seal.

This module offers utilities function for wtg-seal, mainly focused on file parsing.

wtgseal.utils.calc_weights(freqs: Counter[T]) → Counter[T][source]¶

Calculate weights for locust tasks.

Calculate locust task weights based on the frequency that each document was requested, as given by freqs.

Parameters:	freqs ({Counter}) – The number of requests for each document.
Returns:	The weight for each document when mapped as a locust task.
Return type:	Counter

wtgseal.utils.count_requests(file: TextIO) → Counter[T][source]¶

Count the number of requests for each document.

Count the number of requests made for each document based on a given sequence of document requests.

Parameters:	file ({TextIO}) – A file containing a sequence of integers representing indexes of documents in a web server.
Returns:	A collection.Counter with the frequencies for each document.
Return type:	Frequency

wtgseal.utils.parse_documents(file: TextIO) → Generator[List[str], None, None][source]¶

Parse a text file containing documents definitions into lists.

Read a file containing web documents representations and generates lists of URIs related to each object.

Parameters:	file ({TextIO}) – A file handler to the file to be parsed. The file should contain one or more lines, which being a space-separated list of integers. Each line represents a document to be retrieved from a web server. A document is made of one or more files, given by the integers.
Yields:	List[str] – The representation of the next document, i.e. the URIs of the files that compose the document.

Notes

Ideally, file is the file objout.txt generated by the program objects, part of SURGE [1]_.

References

[1]	Barford, P., & Crovella, M. (1998, June). Generating representative web workloads for network and server performance evaluation. In Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems (pp. 151-160).

wtgseal.utils.parse_requests(file: TextIO) → Generator[int, None, None][source]¶

Parse a text file containing a sequence of documents requests.

Read a file containing a sequence of documents identifiers which was to be requested to a web server.

Parameters:	file ({TextIO}) – A file handler to the file to be parsed. The file should contain one or more lines with an integer in each of them.
Yields:	int – The index of the next document to be requested.

Notes

Ideally, file is the file name.txt generated by the program lru, part of SURGE [1]_.

References

[1]	Barford, P., & Crovella, M. (1998, June). Generating representative web workloads for network and server performance evaluation. In Proceedings of the 1998 ACM SIGMETRICS joint international conference on Measurement and modeling of computer systems (pp. 151-160).

Module contents¶

A Web Traffic Generator based on SURGE, Statistics and Locust.

WTG-SEAL is a web traffic generator based on a statistical approach inspired by SURGE and implemented using Locust.