This guide will take you through creating your own levels with Athena CTF, including all of the tools, templates, and classes that have been provided.

• About the Routes
• Creating a New Page or Game
• Templates, Classes, and Functions
• LLM Configuration
• Deploying

Instructions

The instructions route, present at /games/{name} or /{name} such as /games/welcome_game or /how_to is just the default GET route that can contain some instructions for the game, or just be a regular webpage. The instructions will just create a default route at the name of the page, and you can use a current two panel template such as the how_to.html, or create your own.

Verify

The verify route, meant only for games at /games/{name}/verify, such as /games/welcome_game/verify, is a POST request that returns a success or failure dictionary. This return is than evaluated by the Page class, and returns the success function if success is True. This success function will than serve the user with the success message, as well as credit them in the database with passing the level.

Hint

The hint route meant only for games at /games/{name}/hint, such as /games/welcome_game/hint, and only accessible on the website from the frontend route (below), is a POST request that will return a dictionary with the hint text using the "payload" key. This hint will be the default hint if the LLM is not engaged, or there is no database setup, otherwise it will be an LLM generated hint based on past user requests.

Frontend

The frontend route is not a required route, nor is it present in the Page class, as there is no useful default setup for this function. This route is important because it is the only route where hints will be shown, and the hints route will be requested by the frontend. It is intended to be used as some sort of real world interface, from a login page, to a "Post Here!" button that makes a web request to the backend. This is where the challenge is supposed to take place.

Other

You are free to create any more routes that you think would make the user experience better. These routes could be places to make the experience more interactive, or can simulate web requests. You can see how to do this in the tutorial section of this wiki.

A note about all routes

All routes require a cookie to access. This cookie is what differentiates users, and allows for a variety of functions. If a GET request is made to a route without a cookie, the user will be redirected to the home page and assigned a cookie. If a POST request is made to a route without a cookie, the user will receive a malformed request status code, as well as a message to include a cookie.

Configurations and Environments

This application relies on two forms of application configuration, a config.py that sets certain variables depending on whether or not the user has debug mode enabled, and the .env that has all of the secrets for the application. The config.py has the following variables:

Debug

• VERIFICATION_INPUT: Shows an HTML file that has an input box for the flag, and the JS code to accompany submitting the flag to the verify endpoint. Disable if you want the user to submit their own manual post requests
• PARAMETERIZE: Weather or not to parameterize the flag, even if the parameterize function is called. This is so that application function can be altered without altering the code of the level itself.
• GAMES: A dictionary that is initialized as dict(). This stores game data when the application is first created, and is what makes the application so dynamic. This should be left as is.
• URL: This is the base URL for your application. It is used for all of the routes and redirects, so if it is set wrong the application will not function. In debug mode the default URL will work, and your app should be deployed on the same port as is in the URL
• TEMPLATE: The name of the template HTML file for each of your games. This template should be in src/static/templates
• LLM: Toggle for LLM integration. If this is set to True, your LLM key in .env must be set.
• MONGO: Toggle for Mongo integration. If this is set to True, your Mongo URL and database name must be set in the .env
• BASE_DIR: Does not need to be changed, resolves to the "app" directory for the finding of solutions and prompts
• SOLUTION_EXTENSION: The extension of your solution files so that you can have multiple kinds of files in your solutions folder
• ADMIN_CODE: Weather or not to require an admin code to access the deployed site and get a cookie. If this is enabled, users must have an admin code assigned to them at user creation (use admin portal)

Production

Allows for the overwriting of URL, LLM, and MONGO, for easy deploying from development. You could also toggle the template if you desire.

Environment (dotenv)

• AES_PASSWORD: String of text that is used for the encryption password and by extension parameterization
• KEY: LLM token
• MONGODB_URL: Connection string for MongoDB
• DATABASE_NAME: Name of your database in MongoDB
• SESSION_SECRET: Used for passing application secrets from frontend to backend

Creating your level

Your level must be located in the 'levels' directory that is at the root of the application folder. Later in the instructions your will import your levels from this directory.

Imports

The following are all of the FastAPI imports that you should need to operate the application. * Request is the required argument for any route in FastAPI * Response is a response object for simple responses * Redirect is an object that redirects users to another page * HTMLResponse returns an HTML response object * JSONResponse returns a JSON response object

from fastapi import Request, Response
from fastapi.responses import RedirectResponse, HTMLResponse, JSONResponse

Import the preloaded templates from extensions so we have access to all of our templates.

from app.utils.extensions import templates

For included functions and class imports, please visit the Templates, Classes, and Functions section.

Configuration

Get the configuration, this is based on weather or not the application is created with debug True or False.

config = get_config()

If env variables are needed, (they should not be since they should be used in the helper classes) they can be included with dotenv, and used with os.getenv().

from dotenv import load_dotenv
load_dotenv()
your_variable = os.getenv("YOUR_VARIABLE")

Setting functions and variables

Set the default hint string. This is optional.

default_hint = "This is a default hint"

Create the game class, with the default "is_game" unchanged. This will create a new game as opposed to a new game. We pass the default hint to this game, and the name "Never Trust Your Eyes". This will create the route for the game as "/game/never_trust_your_eyes".

game_class = Page("Never Trust Your Eyes", default_hint=default_hint)

Here we create our own instructions function that returns our own template. Button HTML is generated with the generate_button function, where we pass the text that is in the button, and the URL that the button will redirect the user to. We than return the template to the user with the following arguments: * name: The template that we are returning to the user. This specific template is defined in the config as game_template.html. * request: A mandatory argument from FastAPI, just pass the mandatory request variable of the function. * context: The text that is being passed to the template, please visit the Templates, Classes, and Functions section for possible arguments. * status_code: Optional status code to pass, 200 is success.

async def instructions(request: Request):
    text = ("You've been tasked with pen testing a companies backend API. You know this company has a history "
                    " of using the bad practise of 'Security through Obscurity'. "
                    "Click the button below and look around "
                    "for anything that could grant you access. The endpoint is in perfect working order!")

    button = generate_button("Frontend", f"{config.URL}{game_class.url_prefix}/frontend")

    return templates.TemplateResponse(name=config.TEMPLATE, request=request, context={"header": game_class.name,
                                    "text": text, "primary_button": button}, status_code=200)

Instead, you could also just create instruction text, and it will automatically return the default template with the text in the "text" field of the template.

instructions_text = ("You've been tasked with pen testing a companies backend API. You know this company has a history "
                    " of using the bad practise of 'Security through Obscurity'. "
                    "Click the button below and look around "
                    "for anything that could grant you access. The endpoint is in perfect working order!")

Create a super simple verify function that checks the request headers and sets success to True or False:

async def verify(request: Request):
    return {"success": request.headers.get("Authorization") == "YouWillNeverGuessMe"}

You could also create a more complex verify function that customizes your error message and code:

async def verify(request: Request):
    if request.headers.get("Authorization") == "YouWillNeverGuessMe":
        return {"success": True}
    else:
        return {"success": False, "error": {"text": "Unauthorized", "code": 403}}

You than set these functions in the class so they can be used in the route. If the functions are not set the routes will not be created, and the game will not work properly.

game_class.set_functions(instructions=instructions, verify=verify)

Additionally, if you have set instructions as text instead of a function, it works the same way:

game_class.set_functions(instructions=instructions_text, verify=verify)

Set game as the game_class.route. This makes it more simple to add routes, and shorter inputs in the init to create the route.

game = game_class.route

Create your own routes

Create a frontend route that passes the header for the user to sniff and exploit the level. The frontend is a special route that will show the hint button, and will allow the user to get hints. Other routes will not.

@game.get('/frontend', response_class=HTMLResponse)
def check_headers(request: Request):
    return templates.TemplateResponse(request=request, name=config.TEMPLATE, status_code=403, headers={"Password": "YouWillNeverGuessMe"})

More complex frontend functions can be created from the other template arguments such as the one below. This frontend function takes arguments, sets cookies, and uses custom functions for the specific level to create its own return object with objects such as a table.

@game.get('/frontend/{page_num}', response_class=HTMLResponse)
async def serve_frontend(request: Request, page_num: str):
    cookie = request.cookies.get("cookie", None)
    return_text = "You do not have a cookie set, please refresh the page"
    if cookie is None:
        response = templates.TemplateResponse(request=request, name=config.TEMPLATE, context={"header": return_text})
        response.set_cookie("cookie", generate_cookie())
        return response
    else:
        stripped, prepended, appended = get_page(cookie)
        if stripped == page_num:
            return_object = generate_html_text(complete=True, prepended=prepended, appended=appended)
        else:
            logger.log(f"{page_num} is not {stripped}")
            return_object = generate_html_text()

    return templates.TemplateResponse(request=request, name=config.TEMPLATE, context=return_object)

Adding your level

Now that you have created your level, you need to include it in the level group that the framework will present to the user. This is done by editing the 'my_pages.py' file that is located at the root of the application directory.

You must import the Page class of your level into this file. For example:

from levels.welcome_game import welcome_game

In the 'my_pages' function, you are going to append your import to the my_games_list object.

my_games_list.append(welcome_game)

There are comments and examples throughout this file to show you the proper way to import and append.

Creating solution file

In order to allow users to access LLM driven hints, you must provide a solution for your level. This solution can either be programmatic, or an explanation on how to solve the level. Your solution file must be included in the 'solutions' folder that is at the root of the application folder, and must have an identical name to your level. When you first name your level, a replacement operation is performed to turn it into a URL and filesafe string that is used to identify your level throughout. This string will be lowercase with all spaces replaced with underscores and can be found by visiting the level using your browser and looking in the hyperlink. You could also print out the game.url_prefix in your code to get a Python printed string of your level name. The extension to this file should be in your SOLUTIONS_EXTENSIONS in your config.py and the order in which these extensions appear is the order in which they will be provided to the LLM. For example, if you have .txt and .py in your solution extensions, and you have two solutions for one level, the .txt solution will be provided to the LLM as the context, with the .py not being provided. This is so certain file extensions could be provided for certain levels, and others for other difficulties. The solution text will be saved into memory for the first call, and is cached for each hint call after that. This solution will also be inserted into the prompt, so ensure that you have the sanitization required, and there are explanations throughout such as comments.

Templates

game_template.html

Arguments

All arguments are optional but passing no arguments will result in an empty page. All arguments are HTML safe, meaning you can create your own HTML for each passed argument

• header: The header text on the left side of the page. Classes: display-5 fw-bold text-body-emphasis
• table: Table displayed below the header. Classes: None
• text: Regular text. Classes: lead mb-4
• form_data: Form data displayed allowing for forms such as login pages.
• result_text: Text with the id of result. This can interact with buttons. Classes: lead mb-4 id: result
• subtext: Slightly greyed out subtext. Class: mb-4
• hidden_text: Allows for the injection of anything into the template. Meant for hiding text in the page that users can find.
• primary_button: Blue primary button in a div for centring. Works with create_button function, but you could create your own HTML
• secondary_button: Grey secondary button in a div for centring. Will display to the right of primary button in line when both are present
• right_content: All the content that appears below the "Hint" header tag. You can find the default hint HTML in the /hint route in the Page class
• scripts: Inject the JS scripts needed for clientside communication into the page. This is done through the page class with game_class.load_scripts and game_class.scripts["script_name"]

game_template_verify.html

Same arguments as game_template.html, but it will also include an input form for submitting flags, and the JS code to allow the form to function. These files are separate so that if the designer wishes to obfuscate the verify route, they may without revealing its source in the client side code.

how_to.html

This template takes no arguments. Custom made for the how to page.

welcome.html

This template takes no arguments. Custom made for the welcome page.

Classes

Page

This is the main class that creates each page on the website. Other than the root page, every page on the example site is generated with this class.

Arguments

• name: str: Name of the level. The name creates the URL endpoint through lowering it, and replacing spaces with _.
• instructions: string or function: Instructions as a string to be put in the basic instructions function, or define your own instructions HTML and return your own template
• verify: function: A function that returns a dict with a success flag like {"success": True} or {"success": False}. If success is false, you can also return a nested error dict such as {"success": False, "error": {"text": "Unauthorized", "code": 403}}
• scripts Dict[str, str]: A dictionary of js files as values, and your name (does not have to match filename) for the file. This allows you to define all the scripts once per route, and use them in FastAPI context like:

return templates.TemplateResponse(request=request, name=config.TEMPLATE, 
    context={"scripts": game_class.scripts["connection"],
    "header": "Post Here!", "primary_button": button})

• default_hint: str: The default hint string if there is no AI or database hookup

• is_game: bool: This changes the route URL for the route that is created. If is_game is true the route is /games/{name}, otherwise its just /{name}

• filename: Filename (without extension) for the LLM hint. This is optional and will default to the string name

Functions

• set_index: Setter method to set class index, used in route creation
• set_functions: Sets the instruction and verify functions if they were not ready by the time Page creation was needed, for example if a route string needed to be accessed to create verify or instruction functions:

    button = generate_button("Frontend", f"{config.URL}{game_class.url_prefix}/frontend")

• success: Registers in the database that the user has completed the level, and returning the user a success message, along with the URL of the next game
• load_scripts: Setter method for the scripts variable in the class
• get_scripts: Gets you the script that you loaded with load_scripts. Can use this instead of a dictionary lookup This class takes the arguments of name (string), instructions (string or function), verify (function), scripts()

Database Config

This is the class that creates the connection to MongoDB for user of tracking user progress, and enabling AI hints. This simple class reads in the MongoDB URL provided in the .env, and the name of the database where user and level data is stored. This class also makes a call to initiate the Beanie database, therefore setting up the User document defined in the next class.

User Document

The User document, as well as its helper base models (Hint and Level), create an efficient and simple way for the application to interact with MongoDB. All indexes, both unique and sorting, are automatically created with the first interaction with the Mongo collection, meaning the only thing developers need to do is create a database, add the URI and the database name to the env, and {NAME} will create the collection and index's. If you would like to add more data to the collection, or collect less user data, it is as easy as adding or removing fields from the User class.

Functions

This class contains a variety of helper functions for interacting with the table.

Class Methods

These take a cookie string or a User dictionary * get_user: Get the user object from a user cookie string * get_user_by_code: Get the user object from their user access code * generate_leaderboard: Generate the HTML for a leaderboard of all users in the database * create_level: Creates a fresh Level object and returns it * upsert_hint: Upserts a hint for the current level from a cookie string * create_user: Creates a user object from a pre generated cookie string

Self Methods

For these, objects of User must be passed, not just a dict * get_hint_length: Gets the number of hints the user has got for the current level, from the Level string * complete_level: Sets level_completed to True from a level string, from a user object * upsert_hints_user: Upserts a hint from the current level from a level string from a Level string * get_last_hint: Gets the last from the hint string * is_complete: Checks if a level is complete by passing the Level string

SHA256Service

Functions

• hash: Hash a string

AESService

Functions

• encrypt: Encrypt bytes and return a string
• decrypt: Decrypt encrypted bytes

Parameterization

Handles parameterized flags based on the inputted user cookie and any other arguments

Functions

• parameterize_flag: Takes in a user cookie (string) and any other arguments and uses them to create a parameterized flag
• parameterize_with_data: Takes in the user cookie and the data in dictionary form you would like to hide (and later access)
• get_data_from_parameterization: Takes in the encrypted data and decrypts it, returning you a dictionary object of the data

Table

Creates the HTML for a table from a dictionary or list of data, with an optional separate header list.

LLM Service/Manager

Handles all functions when it comes to the LLM interaction. The developers of new levels should use llm_manager.py to get the instances of the LLM that they want via the config file. If a developer wants to add another possible model in, such as Ollama integration, the llm_service.py should be changed to allow for this new model to be called from the manager.

• get_llm: Get the instance of the LLM class of your choice based off of the config
• get_hint: Get a hint based on the prompt and the users past queries

Functions

• get_html: Returns the table HTML

Functions (in extensions.py)

• get_cookie: Returns the user cookie if there is one, otherwise return None
• generate_user_cookie: Generates a new user cookie (made from uuid4), creates a new user from that cookie, and returns that new cookie
• quote_string: Wrap a string in double quotes (for HTML generation)
• generate_button: Generate the HTML for primary and secondary buttons with optional links and element ids with bootstrap classes

The LLM integration for this project is done through two files, an llm_service.py and llm_manager.py, both of which are found in application/app/utils.

LLM Service

The LLM service file utilizes an abstract class to create connectors to various LLM APIs to make the setup process extremely simple. For each of the classes, they have a single public function, get_hint with identical signatures. This function takes the prompt which is constructed from the llm_prompt.txt file, as well as the solution to the level defined in the solutions folder. It also takes the past requests that the user made as a list of strings. From here each function performs the actions required to get a text response from the model and forward it back to the Page class so it can be returned to the user for a hint.

LLM Manager

To avoid any kind of messy setup functions, the LLM manager file contains two functions, get_llm and _create_llm. The create LLM function contains all of the code required to determine which LLM the developer has set in the config, as well as set up that LLM. This function has an LRU cache with a max size of 1 so that when the get_llm function is called, it will only run the function body on first call. This way, the same instance of a class can be used for all LLM calls since all of the get_hint functions are async.

Contributing New Models

If you would like to contribute other models to the service, or just fork for your own use, the process would be as follows:

Config and .env

Create a variable for your new LLM key in the .env file, and change the LLM variable in config.py to your own.

Create a new class in llm_service.py

This class should extend the LLMConnector and should follow a similar signature to the other classes. Set up the self.client using the library of your choice, just ensure that it supports async, or set up your own thread-locking similar to the Ollama class. If you need to add any helper functions to the class, please make them private (preceded with the _). Than you can create your get_hint function from the abstract method in LLMConnector. This should implement whatever API calls are necessary to get the response from your connected LLM.

class MyNewConnector(LLMConnector):
    def __init__(self):
        super().__init__()
        self.client = MyAsyncLLM(
            api_key=os.getenv("MY_NEW_LLM_KEY"),
        )

    async def get_hint(self, prompt, past_queries: Union[List[str], None]=None):
        input_data = [{"role": "system", "content": prompt}]
        if past_queries is not None:
            for query in past_queries:
                input_data.append({"role": "user", "content": query})

        response = await self.client.chat(
            input = input_data
        )
        text = response.text

        return text

Edit llm_manager.py

Change the _create_llm function to include an elif statement for the name of your LLM, or at least what you have decided to call it in the config. For example:

    elif llm_type == "mynewllm":
        return MyNewConnector()

Now when you call get_llm anywhere in your project, your class will be used to get a hint.

Docker

A Dockerfile has been provided for you to simply deploy, both for production and for debug, in a containerized environment. The Dockerfile will by default set the environment variable "DEBUG" to 1, meaning true, so it will by default run the application with Uvicorn instead of Gunicorn, and will run with the development version of the config. You can change this environment variable with the commands shown below.

To build the Docker container, you can run:

docker build -t {container_name} .

Internally, the Docker container will run the application on port 8000. You can forward this port to whatever port on the local machine that you would like, but in the demos below, it is forwarded to local port 8000. The internal Docker port can be changed by altering the start_server.sh script, and altering the bind and port fields.

For running in debug mode, you can run:

docker run -p 8000:8000 -e DEBUG=1 {container_name}

or

docker run -p 8000:8000 {container_name}

Since the default debug is set to true.

For running in production mode, or just your debug=False mode, you can set the debug flag to 1 when running the container:

docker run -p 8000:8000 -e DEBUG=0 {container_name}

Local

The same file that Docker uses to deploy in the Docker container can be used to deploy locally as well. One extra step that must be completed is the installation of the Python packages required to run the application. This can be done by navigating into the application directory and running:

pip install -r requirements.txt

It is recommended to do this in a virtual environment. After installing the Python packages, you can run the shell script that will start the application. If you would like to run in production mode, you can set the DEBUG environment variable to 0. However, it is recommended to set the bind address to a Unix socket if deploying to the internet through a reverse proxy such as NGINX, and the start_server.sh is mainly meant to be used by Docker.