add doxygen documentation

This commit is contained in:
Robert Jeutter 2021-09-10 09:55:51 +02:00
parent 078aedef1e
commit 24b5baf73b
11 changed files with 2433 additions and 0 deletions

2011
doc/Doxyfile.in Normal file

File diff suppressed because it is too large Load Diff

34
doc/bibliography.bib Normal file
View File

@ -0,0 +1,34 @@
% Encoding: UTF-8
@manual{pakkanen_meson_build_system,
author = "Jussi Pakkanen",
title = "The Meson Build System",
url = "https://mesonbuild.com/Getting-meson.html",
}
@Misc{mellanox_nic_benchmark,
title = {Mellanox NICs Performance Report with DPDK 20.05},
howpublished = {\url{https://fast.dpdk.org/doc/perf/DPDK_20_05_Mellanox_NIC_performance_report.pdf}},
note = {Zugriff: 2021-05-17},
}
@Misc{intel_nic_benchmark,
title = {DPDK Intel NIC Performance ReportRelease 20.02},
howpublished = {\url{http://fast.dpdk.org/doc/perf/DPDK_20_02_Intel_NIC_performance_report.pdf}},
note = {Zugriff: 2021-05-17},
}
@Misc{ryzen_benchmark,
title = {Ryzen Website},
howpublished = {\url{https://www.amd.com/en/products/cpu/amd-ryzen-9-5900x}},
note = {Zugriff: 2021-05-17},
}
@Comment{jabref-meta: databaseType:bibtex;}
@Misc{testFab,
AUTHOR = "Menghao Zhang, Gunanyu Li, Shicheng Wang, Chang Liu, Ang Chen, Hongxin Hu, Guofei Gu, Qi Li, Mingwei Xu, Jianping Wu. Poseidon",
TITLE = "Mitigating Volumetric DDoS Attacks with programmable Switches",
YEAR = "2000",
HOWPUBLISHED = "Website",
NOTE = "Aufgerufen 11.05.2021,von https://www.ndss-symposium.org/wp-content/uploads/2020/02/24007-paper.pdf"
}

7
doc/doxygen-extra.css Normal file
View File

@ -0,0 +1,7 @@
div.textblock {
margin: 50px 0 25px 0 !important;
}
h1 {
padding: 100px !important;
}

126
doc/getting_started.md Normal file
View File

@ -0,0 +1,126 @@
Getting started
============
To set up your system as middlebox, just use our installation script `sh install-requirements.sh`.
If you want to do everything by hand you can use the following guide.
If you already installed all requirements, you can go further to [Usage](/usage.md)
#### Install required packages
Just use your command line
```sh
sudo apt-get update -y
sudo apt-get -y -q install git doxygen hugepages build-essential \
linux-headers-`uname -r` libdpdk-dev libmnl0 libmnl-dev libkmod2 \
libkmod-dev libnuma-dev libelf1 libelf-dev libc6-dev-i386 autoconf \
flex bison libncurses5-dev libreadline-dev python3 graphviz \
python3-pyelftools libboost-all-dev plantuml
pip3 install --user meson ninja
uname -r #Kernel >= 3.16 required
ldd --version #glibc >= 2.7 required
```
Install required [sparsehash](https://github.com/sparsehash/sparsehash) Densemap
#### install DPDK
1. Download from [DPDK Website](https://github.com/DPDK/dpdk)
2. extract: tar xf dpdk.tar.gz
3. into folder: cd dpdk
4. build lib, driver, test:
```sh
meson build # include all examples: meson -Dexamples=all build
ninja -C build
```
5. include dpdk to path
- only for session: `export PATH=$PATH:</path/to/file>`
- permanently add `export PATH=$PATH:</path/to/file>` to your `~/.bashrc` file (at end of file).
or with apt `sudo apt install dpdk`
#### Hugepages
1. reserve Hugepages (in runtime):
1. configure Hugepages
`echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages`
`echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages`
2. create directory for mounting
`mkdir -p /mnt/huge`
`mountpoint -q /mnt/huge`
`mount -t hugetlbfs nodev /mnt/huge`
3. Check hugpages `grep Huge /proc/meminfo`
2. Run poll mode driver test: `build/app/dpdk-testpmd -c7 --vdev=net_pcap0,iface=eth0 --vdev=net_pcap1,iface=eth1 -- -i --nb-cores=2 --nb-ports=2 --total-num-mbufs=2048`
##### in boot time
Modify Linux boot time parameters inside `/etc/default/grub`. Huge pages will be spread equally between all NUMA sockets.
`GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=32"`
Update the grub configuration file and reboot.
```sh
grub2-mkconfig -o /boot/grub2/grub.cfg
reboot
```
Create a folder for a permanent mount point of hugetlbfs `mkdir /mnt/huge`
Add the following line to the `/etc/fstab` file: `nodev /mnt/huge hugetlbfs defaults 0 0`
Persistent huge pages are never swapped by the Linux kernel!
#### Mac, IP & PCI adress
1. `ifconfig -a` to view MAC and IP adress
2. `ethtool -i <interface name>` to get PCI adress
3. Load ibuverbs on each reboot with `modprobe -a ib_uverbs`
##### Test your DPDK installation
TX side
```sh
testpmd \
-l <core-list> \
-n <num of mem channels> \
-w <pci address of the device you plan to use> \
--vdev="net_vdev_netvsc<id>,iface=<the iface to attach to>" \
-- --port-topology=chained \
--nb-cores <number of cores to use for test pmd> \
--forward-mode=txonly \
--eth-peer=<port id>,<receiver peer MAC address> \
--stats-period <display interval in seconds>
```
RX side
```sh
testpmd \
-l <core-list> \
-n <num of mem channels> \
-w <pci address of the device you plan to use> \
--vdev="net_vdev_netvsc<id>,iface=<the iface to attach to>" \
-- --port-topology=chained \
--nb-cores <number of cores to use for test pmd> \
--forward-mode=rxonly \
--eth-peer=<port id>,<sender peer MAC address> \
--stats-period <display interval in seconds>
```
#### Problems at Crash
DPDK won't free hugepages and can therefore not be restartet with no hugepages free in system.
Issue could be like you removed `/mnt/huge` and trying to modify the nr_hugepage
```sh
echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages #Numa case
echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages #non-NUMA case
```
steps to follow while un-mapping hugepage:
1. `ls -l /mnt/huge/ `
2. `rm -rf rtemap_*` if there are any "rtemap_*" delete all
3. `mount | grep huge`
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
4. `ls /dev/hugepages`
```sh
rtemap_0 rtemap_1202 rtemap_1408 rtemap_1613 rtemap_1819 rtemap_2023 rtemap_387 rtemap_592 rtemap_798
rtemap_1 rtemap_1203 rtemap_1409 rtemap_1614 rtemap_182 rtemap_2024 rtemap_388 rtemap_593 rtemap_799
```
5. If above files are present delete all of them.
1. `sudo umount /mnt/huge`
2. `sudo rm -R /mnt/huge`
3. Then write 0 to nr_hugepages mentioned at start

BIN
doc/img/doxygen_logo.png Normal file

Binary file not shown.

After

Width:  |  Height:  |  Size: 5.7 KiB

117
doc/installation.md Normal file
View File

@ -0,0 +1,117 @@
Getting
============
#### Install required packages
Just use your command line
```sh
sudo apt-get update -y
sudo apt-get -y -q install git doxygen hugepages build-essential \
linux-headers-`uname -r` libdpdk-dev libmnl0 libmnl-dev libkmod2 \
libkmod-dev libnuma-dev libelf1 libelf-dev libc6-dev-i386 autoconf \
flex bison libncurses5-dev libreadline-dev python3 graphviz \
python3-pyelftools libboost-all-dev plantuml
pip3 install --user meson ninja
uname -r #Kernel >= 3.16 required
ldd --version #glibc >= 2.7 required
```
#### install DPDK
1. Download from Website
2. extract: tar xf dpdk.tar.gz
3. into folder: cd dpdk
4. build lib, driver, test:
```sh
meson build # incl all examples: meson -Dexamples=all build
ninja -C build
```
5. include dpdk to path
- only for session: `export PATH=$PATH:</path/to/file>`
- permanently add `export PATH=$PATH:</path/to/file>` to your `~/.bashrc` file (at the end).
or with apt `sudo apt install dpdk`
#### Hugepages
5. reserve Hugepages (in runtime):
1. configure Hugepages
`echo 64 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages`
`echo 32 > /sys/devices/system/node/node0/hugepages/hugepages-1048576kB/nr_hugepages`
2. create directory for mounting
`mkdir -p /mnt/huge`
`mountpoint -q /mnt/huge`
`mount -t hugetlbfs nodev /mnt/huge`
3. Check hugpages `grep Huge /proc/meminfo`
6. Run poll mode driver test: `build/app/dpdk-testpmd -c7 --vdev=net_pcap0,iface=eth0 --vdev=net_pcap1,iface=eth1 -- -i --nb-cores=2 --nb-ports=2 --total-num-mbufs=2048`
##### in boot time
Modify Linux boot time parameters inside `/etc/default/grub`. Huge pages will be spread equally between all NUMA sockets.
`GRUB_CMDLINE_LINUX="default_hugepagesz=1G hugepagesz=1G hugepages=32"`
Update the grub configuration file and reboot.
```sh
grub2-mkconfig -o /boot/grub2/grub.cfg
reboot
```
Create a folder for a permanent mount point of hugetlbfs `mkdir /mnt/huge`
Add the following line to the `/etc/fstab` file: `nodev /mnt/huge hugetlbfs defaults 0 0`
Persistent huge pages are never swapped by the Linux kernel!
#### Mac, IP & PCI adress
1. `ifconfig -a` to view MAC and IP adress
2. `ethtool -i <interface name>` to get PCI adress
3. Load ibuverbs on each reboot with `modprobe -a ib_uverbs`
##### TX side
```sh
testpmd \
-l <core-list> \
-n <num of mem channels> \
-w <pci address of the device you plan to use> \
--vdev="net_vdev_netvsc<id>,iface=<the iface to attach to>" \
-- --port-topology=chained \
--nb-cores <number of cores to use for test pmd> \
--forward-mode=txonly \
--eth-peer=<port id>,<receiver peer MAC address> \
--stats-period <display interval in seconds>
```
##### RX side
```sh
testpmd \
-l <core-list> \
-n <num of mem channels> \
-w <pci address of the device you plan to use> \
--vdev="net_vdev_netvsc<id>,iface=<the iface to attach to>" \
-- --port-topology=chained \
--nb-cores <number of cores to use for test pmd> \
--forward-mode=rxonly \
--eth-peer=<port id>,<sender peer MAC address> \
--stats-period <display interval in seconds>
```
#### Problem at Crash
DPDK won't free hugepages and can therefore not be restartet with no hugepages free in system.
Issue could be like you removed `/mnt/huge` and trying to modify the nr_hugepage
```sh
echo 0 > /sys/devices/system/node/node0/hugepages/hugepages-2048kB/nr_hugepages #Numa case
echo 0 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages #non-NUMA case
```
steps to follow while un-mapping hugepage:
1. `ls -l /mnt/huge/ `
`rm -rf rtemap_*` (if there are any "rtemap_*" delete all)
2. `mount | grep huge`
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
3. `ls /dev/hugepages`
```sh
rtemap_0 rtemap_1202 rtemap_1408 rtemap_1613 rtemap_1819 rtemap_2023 rtemap_387 rtemap_592 rtemap_798
rtemap_1 rtemap_1203 rtemap_1409 rtemap_1614 rtemap_182 rtemap_2024 rtemap_388 rtemap_593 rtemap_799
```
4. If abive files are present delete all of them.
1. `sudo umount /mnt/huge`
2. `sudo rm -R /mnt/huge`
3. Then write 0 to nr_hugepages mentioned at start

30
doc/mainpage.md Normal file
View File

@ -0,0 +1,30 @@
AEGIS - AntiDoS
==========
> the open source (D)DoS prevention system
AEGIS aims to defend your system against denial-of-service attacks through efficient user-space packet processing. It is build during a software project at the TU Ilmenau.
### Software Project
This lecture teaches students of computer science and engineering informatics methods and techniques of software engineering. By embedding the activities in the software development process, the knowledge is deepened. The course contains the development of software architecture goals, description approaches of the different models and documents, procedure with the development (processes), decision making, architecture styles/patterns and their quality characteristics, as well as the examination/evaluation of architectures.
### TU Ilmenau
Ilmenau University of Technology is a university of the Free State of Thuringia in Ilmenau. It has five faculties of which one teaches computer science in bachelor
### The Problem with (D)DoS
Denial-of-service attacks are a serious and ever-growing threat.
In the digital age, many systems are connected via the Internet or private networks. Many companies, hospitals and public authorities have become popular targets of attack due to inadequate protective measures and high impact ([infopoint-security.de](https://www.infopoint-security.de/cyber-angriffe-auf-deutsche-krankenhaeuser-sind-um-220-prozent-gestiegen/a26177/)). Such attacks are usually carried out for financial or even political reasons, but rarely for the mere disruption or destruction of the target.
In DoS (Denial of Service) and DDoS (Distributed Denial of Service) attacks, servers and infrastructures are overloaded with a flood of meaningless requests to such an extent that they are prevented from operating normally. This can result in users no longer being able to reach the services offered by the operator and data can be lost in the attack.
In this case, even weak computers can cause great damage to much more powerful recipients. In botnets, the attacks can additionally be coordinated by several computers at the same time, originate from a wide variety of networks ([tecchannel.de](https://www.tecchannel.de/a/trend-micro-latente-gefahr-durch-botnet-sdbot,2024687)) and thus simultaneously increase the attack power and make detection more difficult.
The imbalance between simplicity in generating attacks versus complex and resource-intensive DoS defenses further exacerbates the problem. Although occasional successes are achieved in the fight against DoS attacks (e.g., shutting down some large "DoS-for-hire" websites), the volume of DoS attack data continues to grow. Between 2014 and 2017 alone, the frequency of DoS attacks has increased by a factor of 2.5 and the attack volume is almost doubling every year ([ns-cdn.neustar.biz](https://ns-cdn.neustar.biz/creative_services/biz/neustar/www/resources/whitepapers/it-security/ddos/2016-apr-ddos-report.pdf)). Damage is estimated at between $20,000 and $40,000 per hour worldwide ([datacenterknowledge.com](https://www.datacenterknowledge.com/archives/2016/05/13/number-of-costly-dos-related-data-center-outages-rising)).
In the area of commercial DoS defenses, some approaches have stood out (e.g., [Project Shield](https://projectshield.withgoogle.com/landing), [Cloudflare](https://www.cloudflare.com/ddos/), or [AWS Shield](https://aws.amazon.com/de/shield/)). However, the use of commercial solutions poses some problems, such as sometimes significant costs or the problem of the necessary trust that must be placed in the operator of a DoS defense. Consequently, an efficient defense against DoS attacks with own means is often a desired goal - especially if several systems can be protected at the same time.
The goal of this software project is to create a system between the Internet connection and the internal network that can effectively defend against (D)DoS attacks at a high bandwidth and in continuous operation, while users can still access their services without restrictions. The resulting application implements (D)DoS traffic inspection and an intelligent rule generator, protecting internal networks from external threats that would overload the system. It includes traffic analysis algorithms that can detect and filter out malicious traffic without affecting the user experience and without causing downtime.
#### related links
[DPDK](https://www.dpdk.org/)
[TU Ilmenau](https://www.tu-ilmenau.de)

53
doc/meson.build Normal file
View File

@ -0,0 +1,53 @@
doxygen = find_program('doxygen', required : false)
if doxygen.found()
cdata = configuration_data()
cdata.set('ALLOW_UNICODE_NAMES', true)
cdata.set('CALL_GRAPH', true)
cdata.set('COLLABORATION_GRAPH', true)
cdata.set('CPP_CLI_SUPPORT', true)
cdata.set('DOT_GRAPH_MAX_NODES', 20)
cdata.set('DOT_TRANSPARENT', true)
cdata.set('EXTRACT_ALL', true)
cdata.set('EXTRACT_PACKAGE', true)
cdata.set('EXTRACT_PRIVATE', true)
cdata.set('FULL_PATH_NAMES', false)
cdata.set('GENERATE_TREEVIEW', true)
cdata.set('HAVE_DOT', true)
cdata.set('HIDE_UNDOC_RELATIONS', false)
cdata.set('HTML_EXTRA_STYLESHEET', join_paths(meson.current_source_dir(),'doxygen-extra.css'))
cdata.set('HTML_TIMESTAMP', true)
cdata.set('OUTPUT_DIRECTORY', join_paths(meson.project_build_root(), 'doc'))
cdata.set('PLANTUML_JAR_PATH', join_paths(meson.current_source_dir(), 'plantuml.jar'))
cdata.set('PLANTUML_CFG_FILE', join_paths(meson.current_source_dir(), 'plantuml-config.txt'))
cdata.set('PROJECT_DOCDIR', meson.current_source_dir())
cdata.set('PROJECT_EXAMPLESDIR', join_paths(meson.project_source_root(),'examples'))
cdata.set('PROJECT_INCDIR', join_paths(meson.project_source_root(),'include'))
cdata.set('PROJECT_LOGO', join_paths(meson.current_source_dir(), 'img', 'doxygen_logo.png'))
cdata.set('PROJECT_NAME', meson.project_name())
cdata.set('PROJECT_SRCDIR', join_paths(meson.project_source_root(),'source'))
cdata.set('PROJECT_TESTDIR', join_paths(meson.project_source_root(),'test'))
cdata.set('RECURSIVE', true)
cdata.set('SOURCE_BROWSER', true)
cdata.set('TEMPLATE_RELATIONS', true)
cdata.set('TREEVIEW_WIDTH', 200)
cdata.set('UML_LIMIT_NUM_FIELDS', 100)
cdata.set('UML_LOOK', true)
cdata.set('USE_MATHJAX', true)
cdata.set('VERSION', meson.project_version())
doxyfile = configure_file(input: 'Doxyfile.in',
output: 'Doxyfile',
configuration: cdata,
install: false)
docdir = join_paths(get_option('datadir'),'doc', meson.project_name())
html_target = custom_target('doc',
build_always_stale: true,
input: doxyfile,
output: 'html',
command: [doxygen, doxyfile, meson.project_source_root()],
install: true,
install_dir: docdir,
build_by_default: true)
else
warning('Documentation disabled without doxygen')
endif

5
doc/plantuml-config.txt Normal file
View File

@ -0,0 +1,5 @@
skinparam backgroundColor #ffffff
skinparam sequenceArrowThickness 2
skinparam maxmessagesize 60
skinparam ParticipantPadding 20
skinparam BoxPadding 10

BIN
doc/plantuml.jar Normal file

Binary file not shown.

50
doc/usage.md Normal file
View File

@ -0,0 +1,50 @@
# Configuration
### config.json
The following configuration points are to be set in the config.json file in the root
directory of this project.
* `number_of_worker_threads` : Number of threads that should run through the aegis-pipeline in parallel.
* type: string
* Default: the maximum number of threads (aka dpdk-lcores) possible is chosen.
* if you want the default value to be set simply remove this key-value pair from the config.json file.
* Minimum: 1.
* Maximum: Number of available dpdk-lcores (most likely available Threads) on the system minus 1.
### Command Line Options
The following command line arguments are supported:
* `--config -c` => Location to config file. Default => **/etc/aegis/aegis.conf**.
* `--dpdk_version` => print version of dpdk version installed
* `--help -h` => Print help menu for command line options.
* `--ifname` => list all interfaces with name
* `--keep_files` => should generated files be kept after exit
* `--list -l` => List all filtering rules/settings scanned from config file.
* `--meson_bin` => print location of the meson binary
* `--log_level` => set logging level
* `--test_list` => test rules list for errors
Commandline instructions will overwrite the configration file settings!
#### Example
```bash
aegis --keep-files --log_level 3
aegis --ifname
aegis --list
```
### Usage
How to use AEGIS:
#### Compile your own AEGIS
To compile your AEGIS you need to have all requirements installed; refer to [Getting Started](/getting_started.md).
1. Run `meson build` in your root folder of AEGIS. Meson will collect all necesarry files for you.
2. `cd build` into your build folder
3. Run `ninja` to compile all source code to your system specifiy binaries
#### Simple usage
Call `aegis` to run the AEGIS CLI and start your service. The CLI will guide you trough all possibilities.