Contents

Main Features

Last update: Dec 13 24 at 15:03

fenicsR13: A Tensorial Mixed Finite Element Solver for the Linear R13 Equations Using the FEniCS Computing Platform

Pipeline status Test coverage Documentation Website Documentation Website Zenodo Link with DOI

Paper

Lambert Theisen, Manuel Torrilhon. 2021. fenicsR13: A Tensorial Mixed Finite Element Solver for theLinear R13 Equations Using the FEniCS Computing Platform. ACM Trans. Math. Softw. 47, 2, Article 17 (April 2021), 29 pages, 10.1145/3442378. Preprint @ arXiv (Free PDF).

Main Features

  • Solving Steady 2D Linear R13 Equations in Normalized Form

  • Stabilization with: Continous Interior Penalty (CIP) or Galerkin Least Squares (GLS)

  • Option for convergence study with input for exact solution

  • Output in XDMF/HDF5 and PDF

  • Arbitray finite element combinations (thanks to FEniCS)

  • Easy setup of parameter studies

  • Structured/Documented YAML input file format

  • Rarefied gas flow effects predictable: Knudsen paradox, Knudsen pump, …

  • Interface for Gmsh meshes

  • Full exposure of the PETSc preconditioned iterative solver library

Installation

  1. Install Git (if not shipped with you OS).

  2. Install Git LFS (needed to download the meshes from the repository).

  3. Clone and open the repository:

# Clone Repository and open main folder
git clone https://git.rwth-aachen.de/lamBOO/fenicsR13.git
cd fenicsR13

Alternatively: download the repository, unpack and open it.

Docker

It is recommended to use the program with Docker, a virtualization software available (with free docker account) for `download`_ for most operating systems. Our repository provides a Docker container based on the official FEniCS Docker image which contains all necessities for FEniCS and has been adjusted to solve the R13 equations. The program docker-compose is also required.

Starting the Environment

Make sure you installed Docker and it is running on your system. You can start the fenicsR13 compute environment from the root-directory of the repository using the command-line

## 1) FOR USERS:
## - Pulls and runs "fenicsr13_release" service
## - Uses preinstalled latest version of fenicsR13
docker compose pull fenicsr13_release
docker compose run --rm fenicsr13_release

## 2) FOR DEVELOPERS (uncommend the last 3 lines of this block):
## - Builds the "fenicsr13_debug"
## - Install an editable version of fenicsR13
## - The "pip"-command install an editable version of fenicsR13
## - This allows a modification of the source files which can be directly
##   executed within the debug container
# docker compose build fenicsr13_debug
# docker compose run fenicsr13_debug
# sudo pip install -e .

When you run this for the first time, docker will pull (download and extract) the container image from our repository which is roughly 800MB and the download may require some patience. After the initial download the docker image will be stored (2-3 GB) on your system and any new run will start the container immediately.

The above docker command will start a RedHat-based bash environment. The repository is mounted as a volume under /home/fenics/shared in the container and should be the default folder on startup. What ever is changed in this folder or its sub-folders will change in the folder on your original system as well. However, any changes outside this folder will be gone after you exit the docker environment.

Running a Simulation

To execute a simulation case, go to to the case folder (e.g. examples/lid_driven_cavity)

# [It might be required to install the package inside the Docker container]
pip install --user -e .

# Move to folder:
cd examples/lid_driven_cavity

We provide a script to utilize gmsh and generate a H5 mesh-file from a local geometry file by

# Create mesh:
./create_mesh.sh

To run a simulation execute the solver main program fenicsR13.py (which is located in the src-directory in the top level) while specifying an input file as first command line argument.

# Run program with given input file:
fenicsR13 input.yml

Output files will be written to a folder which is named after the output_folder keyword of the input.yml. For immediate inspection the output folder contains simple visualizations in PDF files for each of the fields (temperature, pressure,…).

The numerical results for each field is ouput into h5-files, including mesh data and with corresponding xdmf-file. The XDMF-files can be opened in Paraview to perform visualization, e.g., with Paraview > File > Open > u_0.xdmf > Apply filters

# Leave directory:
cd ../..

Channel Flow Example

We provide a simple example of a flow through a finite-length channel in 2D.

# Move to folder:
cd examples/channel_flow_force
# Create mesh:
./create_mesh.sh
# Run program with given input file:
fenicsR13 input.yml

In the output folder the results can be post-processed to demonstrate the Knudsen paradox in a simple table.

# Go to folder with simulation results (=casename in input.yml)
cd channel_flow_force
# Generate correlation data between Knudsen number and massflow
bash postprocessing.sh
cat table.csv
# Leave directory:
cd ../..

Convergence Study

We can test the convergence of the R13 discretization on a simple double-cylindrical geometry.

# Move to folder:
cd tests/2d_r13
# Meshes are already in Git:
ls ../2d_mesh
# Run program with given input file:
fenicsR13 inputs/r13_1_coeffs_nosources_norot_inflow_p1p1p1p1p1_stab.yml
# Go to folder with simulation results (=casename in input.yml)
cd r13_1_coeffs_nosources_norot_inflow_p1p1p1p1p1_stab
# Open errors:
cat errors.csv

Additional information

Parallel Execution

FEniCS allows simple parallelization using MPI

# Parallel execution ("-u" to flash stdout)
# Usage: mpirun -n <numberOfProcesses> <serialCommand>
# E.g.: mpirun -n 4 fenicsR13 input.yml

Building the Docker Image Locally

The main folder of this repository contains a Dockerfile defining the used environment. Here, we used the optimized and official FEniCS Docker image and include Gmsh and install some requirements from the requirements.txt. This can take a while, especially the Gmsh mirror can be quite slow. To avoid very long execution commands (docker run <..> -v <volume share> <etc..>), a docker-compose.yml is used to store all these parameters. docker compose acts as an wrapper for the Docker execution.

The fenics environment (also called service in the docker-compose.yml) first has to be build and can be executed afterwards. The command to build the container is

# build fenics service
docker compose build fenicsr13_release

Interactive Docker Sessions

It is possible to use a Jupyter sever or a X11 forwarding but this is not recommended anymore. All relevant plots are now written by default without the need for the tricky X11 forwarding or interactive usage with Jupyter.

Documentation

Documentation using Sphinx is available.

Pre-Build Version

Visit the hosted version on Gitlab Pages or download the artifacts from Gitlab’s CI pages-pipeline.

Manual Generation

# cat .gitlab-ci.yml
cd docs
sphinx-apidoc -o source/src ../src
sphinx-apidoc -o source/tests/2d_heat ../tests/2d_heat
sphinx-apidoc -o source/tests/2d_stress ../tests/2d_stress
sphinx-apidoc -o source/tests/2d_r13 ../tests/2d_r13
sphinx-apidoc -o source/tests/3d_heat ../tests/3d_heat
sphinx-apidoc -o source/tests/3d_stress ../tests/3d_stress
sphinx-apidoc -o source/tests/3d_r13 ../tests/3d_r13
sphinx-apidoc -o source/examples ../examples
make html
make latex

Developer Legacy Notes

Developer Tips

  • Monitor the performance of the program with e.g.:

    htop -p `{ fenicsR13 inputs/1_coeffs_nosources_norot_inflow_p1p1p1p1p1_stab.yml > /dev/null & } && echo $!`

  • Use doctest with python3 -m doctest -v src/meshes.py

  • Run pydocstyle once in a while

  • Matplotbib fails when having wrong backend on macOS

    • Fix: Add backend: TkAgg to ~/.matplotlib/matplotlibrc file

  • Performance in Docker is way better than conda build, especially JIT compilation is faster

  • Get C++ inlcude paths: echo | gcc -E -Wp,-v -

  • Bessel functions in DOLFIN:

    • C++17 functions cannpot be used. Boost functions also not per default. Expression("boost::math::cyl_bessel_i(0,atan2(x[1], x[0]))", degree=2) is allowed if one changes in file /usr/local/lib/python3.6/dist-packages/dolfin/jit/jit.py

      _math_header = """
// cmath functions
#include <boost/math/special_functions/bessel.hpp> // Added
%s
"""

Python notes

  • Get current work directory:

    import os
cwd = os.getcwd()
print(cwd)

  • Latex font for matplotlib:

    # LaTeX text fonts:
# Use with raw strings: r"$\mathcal{O}(h^1)$"
plt.rc('text', usetex=True)
plt.rc('font', family='serif')

  • Get system path where modules are searched:

    import sys
print(sys.path)

Create new version tag

  1. Add CHANGELOG entry

  2. Adapt version in conf.py for docs and setup.py for package

  3. Change badge in README.rst

  4. Change version in program information printing

  5. Build new Docker container

Gitlab CI Setup

  • The build stage has to be triggered manually when something in the setup changes. This is because it takes a fair amount of time.

  • In ~/.gitlab-runner/config.toml (for the runner):
  • Run local: gitlab-runner exec docker --docker-privileged build or with build replaced by job name

    • maybe local vars have to be change to use local Docker images because CI_REGISTRY,… are not set

An example gitlab runner config/toml in ~/.gitlab-runner can look like:

concurrent = 1
check_interval = 0

[[runners]]
name = "190716-macbookpro"
url = "https://git.rwth-aachen.de/"
token = "<PRIVATE_TOKEN>"
executor = "docker"
environment = ["DOCKER_TLS_CERTDIR="]
[runners.docker]
    tls_verify = false
    image = "docker:stable"
    privileged = true
    disable_cache = false
    volumes = ["/cache"]
    shm_size = 0
    pull_policy = "if-not-present"
[runners.cache]

Further Installation Tips

Interactive Jupyter Notebooks with Microsoft’s Visual Studio Code

This is may be a convenient solution. Run a file with %run ../../fenicsr13/fenicsr13.py

X11 Window Forwarding on OSX

See guide for the programs to install. Then source the open-macos-gui-tunnel.sh with . open-macos-gui-tunnel. Afterwards, start the container and run the change-matplotbib-backend-tkagg.sh script to set the right matplotlib’s output.

X11 Window Forwarding on Windows

A nice guide can be found here on Dev.to.

The steps can be summarized as:

  1. Install the package manager Chocolatey.

    REM comment: open cmd.exe as admin
@"%SystemRoot%\System32\WindowsPowerShell\v1.0\powershell.exe" -NoProfile -InputFormat None -ExecutionPolicy Bypass -Command "iex ((New-Object System.Net.WebClient).DownloadString('https://chocolatey.org/install.ps1'))" && SET "PATH=%PATH%;%ALLUSERSPROFILE%\chocolatey\bin"

  2. Open cmd.exe as admin and install VcXsrv Windows X Server.

    choco install vcxsrv

  3. Open a X11 server and set the ip variable (that is used in the docker-compose.yml when starting the Docker container to set export DISPLAY=${ip}:0).

    # home of this repo
source sripts/open-windows-gui-tunnel.sh

Contact

Author:
Lambert Theisen
lambert.theisen@rwth-aachen.de
Supervisor:
Prof. Dr. Manuel Torrilhon
Lehrstuhl für Mathematik (MathCCES)
RWTH Aachen University
mt@mathcces.rwth-aachen.de

Change log

1.4 (2020-11-01)

  • Add convergence study for Knudsen pump

  • Add thermal edge flow example

    • With SLURM script and Amazon EC2 setup script

  • Add timing reports in debug log for assemble, solve

  • Add average velocity postproccessing

  • Update docker-compose by:

    • Remove Git binding

    • Increase SHM size for clusters

  • Improve geoToH5 by using unique tmp names

    • This allows for parallel execution

  • Add feature to evaluate arbitrary line integrals

1.3 (2020-08-18)

  • Add option to use Garlerkin Least Squares (GLS) stabilization

  • Allow for different chi_tilde on boundaries

  • Add option for multiple mesh regions:

    • Allows for different Knudsen numbers in regions

    • Add new example: Lid-driven cavity with two mesh regions

  • Optimization of CI pipeline and scripts in Gitlab:

    • Assert Flake8 compliance

  • Add option to output solution vectors (for model order reduction experiments)

  • Improve README with R13 equation set and list of features

1.2 (2020-02-17)

  • Change system:

    • Change RHS for energy coupling consistency

    • Add delta term

    • Add body force

    • Add div(u) coupling to energy balance (also change RHS for elimination)

    • Rename deltas and chi_tilde

  • Add body froce driven channel flow

  • Add P2P1P1P2P1 R13 validation case

  • Revert to MUMPS solver

1.1 (2019-12-15)

  • Symmetrize system

    • Scale the equations to match

    • Introduce subfunctionals

    • Subfunctionals are equal for off-diagonal entries

    • Resolve stf-terms using orthogonality

    • Make r13 as default in formulation rather than decoupled

    • Refactor CIP as separate subfunctionals

    • Add antisymm ring, not used for now but can improve convergence

  • Create package

    • Move files into separate folder and add setup.py

    • CI is changed

    • Installation through “pip install .” in toplevel

  • Clean repository

1.0 (2019-09-23)

  • Improve examples

    • Remove auxiliary files, clean cases, improve output plots and change some meshes

  • Add WELCOME screen to Docker container with link to website and documentation

  • Introduce more variables as dolfin.Constant() for parameters study w.o. compiling

  • Add time measure for linear solve

  • Thesis submission version

0.5 (2019-09-13)

  • Add more examples with tests and documentation:

    • Lid-Driven Cavity

    • Channel Flow with Knudsen paradox plot

    • Knudsen pump

  • Add option to perform parameter studies

  • Add massflow reporting option for arbitrary BCs

  • Fix P1P2P4 stress test case

  • Add more printing statement to program output

  • Change formulation:

    • Rename gamma to epsilon in inflow model

    • Rename tau to Knudsen to have real dimensionless equations

    • Replace sym(psi) -> psi because symmetric per definition

    • Fix stf3d2 for arbitrary

  • Extend documentation:

    • Extended tutorial

    • Move legacy notes to bottom of README

0.4 (2019-08-21)

  • Finish documentation

    • Also includes some doctests to test for edge cases

  • Introduce develop branch to only have major version at master branch

  • Add relative error calculation

  • Add channel flow example (to test for Knudsen paradox)

  • Fix error calculation for higher-order Ansatz function

    • The previous error was based on DOFS (P2 elements therefore differ), the new error is based on vertex values

0.3 (2019-08-11)

  • Full linear R13 now converges

  • Inflow model works

  • Restructuration of BC specification

  • Minor improvements in plotting routine

0.2 (2019-07-29)

  • Decoupled stress system converges

  • Add separated tensor operations module

    • This was needed to implement operations on synthetic 3D tensors

  • Add pytests for stress

  • Add new logo

  • Add more Sphinx documentation

  • Restructure repository

0.1 (2019-07-17)

  • Add logo

  • Add Sphinx documentation

  • Add pytests

  • Add Gitlab CI scripts

Indices and Tables

Contents

next

fenicsR13