Skip to content

README.md

Latest commit

 

History

History
218 lines (193 loc) · 8.46 KB

README.md

File metadata and controls

218 lines (193 loc) · 8.46 KB
Raw

TNNT backend powered by Django REST Framework

This is a rewrite of the old Perl backend, intended to be designed as a REST API using Django REST framework. Game data is source from xlogfile entries and stored in a database using Django's object model as DB abstraction. Django REST Serializers provide an abstraction for deserializing xlogfile data and serializing query results before rendering the output as JSON.

Endpoints

  • / - returns dictionary of the form {endpoint}: {endpoint url}
  • /games - all games
  • /ascended - all ascensions
  • /players - list of players
  • /players/<name> - games by player
  • /players/<name> - ascensions by player
  • /leaderboards - list of leaderboards
  • /leaderboards/conducts
  • /leaderboards/minscore
  • /leaderboards/realtime
  • /leaderboards/turncount
  • /leaderboards/wallclock

Dependencies and setup

Arch Linux

$ sudo pacman -S mariadb mariadb-clients python-mysqlclient python-yaml
$ sudo mysql_install_db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
$ sudo systemctl start mysql.service

Debian/Ubuntu

$ sudo apt-get install python3 pip mariadb-server mariadb-client
$ sudo apt-get install python3-dev default-libmysqlclient-dev build-essential

macOS Big Sur

In order for the rest of this guide to be consistent, add the following to ~/.zshrc.

alias python='python3'
alias pip='pip3'

Install mariadb:

$ brew install mariadb
$ mysql.server start

OS-independent steps

Optional: enter python virtual environment before installing dependencies

$ virtualenv env --no-site-packages
$ source env/bin/activate

Install python dependencies

$ pip install -r requirements.txt

Create secrets file

Go to http://www.miniwebtool.com/django-secret-key-generator/ and add add the following to secrets.sh:

export SECRET_KEY='<your_key_here>'
export DATABASE_PASSWORD='<db_pass_here>'

$ source secrets.sh

Setup database

$ sudo mysql

> CREATE DATABASE scoreboard;
> CREATE USER 'tnnt'@'localhost' IDENTIFIED BY '<db_pass_here>';
> GRANT ALL ON *.* TO 'tnnt'@'localhost';

migrate

$ python manage.py migrate
$ python manage.py createsuperuser
$ python manage.py makemigrations scoreboard
$ python manage.py migrate

Import achievements, conducts, and trophies data

Note: this requires a copy of the TNNT game source to pull TNNT achievement names and descriptions from. However, achievements are now part of the repository, and the first command below only needs to be run if/when achievements are modified in the TNNT game source code. The import of fixtures should also only be done once, when a fresh Django server is being set up.

$ ./ach_to_yaml.sh /path/to/tnnt/source > scoreboard/fixtures/achievements.yaml
$ ./manage.py loaddata achievements conducts sources trophies

Serving up static content in production mode

Note: in tnnt/settings.py, when DEBUG = False (production mode), run this command to properly stage static content.

$ ./manage.py collectstatic

Running and operation

The pollxlogs command will read xlog data from a list of sources with associated URLs, saved in the DB. Those sources were defined as fixtures and imported with the above loaddata command.

$ ./manage.py pollxlogs
# alternatively, if you want to load a local xlogfile for testing, do:
$ ./manage.py pollxlogs --file /path/to/xlog

The aggregate command will loop over all games and compute various aggregate data for individual players and for clans standings.

$ ./manage.py aggregate

Optional: point DGL_DATABASE_PATH at real database

This is in tnnt/settings.py. By default, it points at dgamelaunch_test.db, which provides several "registered dgamelaunch accounts" (alice, bob, chuck, david, eve, gimli, janet, omghax, and sally; each password is the same as the username) for dev and testing purposes. For production, you will want to point this to the full path of the real dgamelaunch sqlite database.

Run dev server

$ ./manage.py runserver

Clearing databases

Use reset_db.sh when you want to reset the DB structure; i.e. after changing a model.

Use manage.py wipe_db when you want to delete records from the database. It offers different options for erasing items. Note that all the options wipe all the games and reset the saved file positions of the downloaded source files to 0.

Loading a custom xlogfile

This is if you want to test specific behavior with a manually edited xlogfile. After making sure that Games are cleared from the database and at least one Source exists in the database, run:

$ ./manage.py pollxlogs --file <your custom xlogfile>

Each game is attached to the Source named by its server field (us/eu/au.hardfought.org). Games outside TOURNAMENT_START..TOURNAMENT_END in tnnt/settings.py are skipped.

Test xlogfiles

Three test xlogfiles are provided (test-us.xlog, test-eu.xlog and test-au.xlog). Each holds 3000 real games from the 2025 tournament: 100 ascensions, 300 scummed games, 1950 other games with achievements and 650 without (2025 has only 326 ascensions in all, so that category cannot scale with the rest). The games are renamed to the accounts in the test dgamelaunch sqlite database, three per server: alice/bob/chuck on us, david/eve/gimli on eu, janet/omghax/sally on au (with DEBUG=True the site logs in against that database). Dumplog links do not work for the renamed games. The records are converted to the record format of the current game (version 5.0.0, its achievement and conduct bits, the tnntconduct field), so they decode to the achievements and conducts they had in 2025, less the few the game has since removed.

The games are from the previous tournament, so pollxlogs --file skips them. Load them with load_test_xlogs, which imports them with the tournament window moved to November of the previous year and then runs aggregate. Aggregate counts every game in the database, so take them out again before the tournament starts:

$ ./manage.py load_test_xlogs
$ ./manage.py load_test_xlogs --remove --dry-run
$ ./manage.py load_test_xlogs --remove

Removal deletes the named players' games from before TOURNAMENT_START, then those players unless they have a login, a clan, an invite, a tournament game or a game in progress. On the production server, restart tnntbot afterwards so that it announces those names normally again.

The game moves achievement and conduct bits whenever its lists change, so rebuild the files after every resync of the fixtures, with the virtualenv active:

$ ./create_test_xlogs.py 2025

Any other year's archive works too. A year whose records were written by an older game must be listed in OLD_FORMATS in that script, with the commit whose fixtures decoded that tournament.

Test mode on the production server

test_mode.sh (run as root) puts the live site into test mode, to see how changes look with realistic data, and takes it out again:

$ ./test_mode.sh on --dry-run    # every check, plus a rehearsal that is rolled back
$ ./test_mode.sh on
$ ./test_mode.sh status
$ ./test_mode.sh aggregate       # recompute clan totals, trophies and leaderboards
$ ./test_mode.sh off --dry-run
$ ./test_mode.sh off

on backs up the database, restarts the site with DEBUG=True (set in /etc/default/tnnt_website), so that logins are checked against dgamelaunch_test.db, and runs ./manage.py test_mode on. That command writes down what the database holds, turns non-admin logins and clan operations on and loads the test xlogfiles, in one transaction. off backs up the database, stops the site, runs ./manage.py test_mode off (which deletes everything created since on, including whatever the test players did, and puts the switches back), starts the site without DEBUG and restarts tnntbot. While the site is in test mode only the test accounts can log in, admins included, and error pages show tracebacks to anyone. Turn it off before the tournament starts.

Clan totals, trophies and leaderboards are computed by aggregate, which cron runs every 5 minutes only during the tournament. In test mode, run ./test_mode.sh aggregate after creating or changing clans. It refuses to run while the cron job's update is running.