A Flae8 reporter plugin for CodeGrade AutoTest v2.
This plugin writes messages to CodeGrade AutoTest v2's structured output channel.
For each violation Flake8 reports, a comments
message is written as described
in CodeGrade's documentation. When Flake8 finishes its analysis, a final message
is written with the amount of points that were achieved in the code quality run.
In order to use the custom reporter, make sure you have installed both Flake8 and this package:
python3 -m pip install flake8==6.0.0
python3 -m pip install cg-flake8-reporter
This reporter is registered with Flake8 as cg-flake8-reporter
, to use it run
flake with the option --format=cg-flake8-reporter
.
The custom reporter adds a few new options to Flake8: cg-points-deducted
,
cg-flake8-fd
and cg-base-path
.
The cg-points-deducted
option makes it possible to configure the amount of
points (in percentage) that each violation deducts from the total points.
The cg-points-deducted
expectes a string in the following format as input:
'info:<percentage>,warning:<percentage>,error:<percentage>'
Each of the violation levels must be present in the provided string. The
percentage provided should be an integer number without the %
symbol.
If you wish for a violation level to not deduct points, simply set it to 0
.
The cg-flake8-fd
option makes it possible to configure where the reporter
will write its output. By default, the value is 1
, which means the reporter
will write to stdout
. Within AutoTest v2 it is recommended to use file
descriptor 3
so that the comments will be visible in CodeGrade's UI. The
Flake8 step will already set this up for you.
The cg-base-path
allows you to restrict which files the reporter will report.
For example, if you only want files to be reported within the server
directory
of the student, you may want to set --cg-base-path=~/student/server/
. Beware,
the Flake8 step always sets cg-base-path
to the root of the student's
workspace. If you want to customize this, you should use a Custom Test step
instead.
The cg-code-prefix-mapping
options allows you to create a custom mapping
between error code prefixes and severities. The code prefixes are groups of errors.
The option expects a string in the following format:
'_prefix_:_severity_,_prefix_2_:_severity_2_,...'
The normal mapping can be represented as follows:
'F:error,E:error,W:warning:C:info,N8:info'
The standard mapping will always be set, however you can overwrite existing keys with new values should you want this.
This mapping also allows you to use libraries that introduce new error types, which will use a different, unknown code prefix.
The cg-default-severity
option will set provide a fallback severity, when we
have checked the default mapping, (plus your supplied mapping,) and could not
find any set severity for the code prefix.
For example, when we install a library that adds error codes in the form of
D...
, where the dots are the actual error code numbers. We will search for
an entry in the severity mapping, and fallback to the default if we cannot find
anything.
The default value for this option is: 'unset', which means that there can be no deduction for this type of code, and will not get flagged as categorised errors.
To run Flake8 with the custom reporter:
python3 -m flake8 \
--format=cg-flake8-reporter \
--cg-points-deducted='info:1,warning:3,error:5,unknown:0' \
--cg-flake8-fd=1 \
./
To run flake8 when having a library installed that adds error codes with prefix D
and assign the info
severity to each error starting with a D
:
python3 -m flake8 \
--format=cg-flake8-reporter \
--cg-points-deducted='info:1,warning:3,error:5,unknown:0' \
--cg-code-prefix-mapping='D:info' \
--cg-flake8-fd=1 \
./