Move OTS database from MySQL to MariaDB

If you installed MySQL 8.0 and imported OTS schema.sql with default settings, you database uses encoding not supported by MariaDB.

If you dump database using:

mysqldump dbname > dump.sql

and try to import it into MariaDB using:

mysql dbname < dump.sql

It will show error. To fix this, run these 2 commands:

sed -i 's/utf8mb4_0900_ai_ci/utf8_general_ci/g' dump.sql
sed -i 's/CHARSET=utf8mb4/CHARSET=utf8/g' dump.sql

How to disable CloudFlare IPv6 access

If you want to disable IPv6, you are probably running project with some old code that is not able to process IPv6 REMOTE_ADDR in PHP. I assume you do not use CF API at all, so fastest option will be to use global API key to disable IPv6 and then generate new global API key.
Get your CF account global API key from CF user panel: https://dash.cloudflare.com/profile/api-tokens

Execute in terminal with curl installed (ex. Linux) or Postman:

curl -X PATCH "https://api.cloudflare.com/client/v4/zones/YOUR-ZONE-ID/settings/ipv6" \
-H "X-Auth-Email: YOUR@EMAIL.COM " \
-H "X-Auth-Key: YOUR-GLOBAL-API-KEY" \
-H "Content-Type: application/json" \
--data '{"value":"off"}'

with your ZONE ID, CF E-MAIL and GLOBAL API KEY.

If you execute this curl line on your Linux server without space as start of line, your CF global API key will be stored in Linux command line history. Generate new global API key in CF panel, it will disable old API key.

Getting free VPS from Oracle Cloud

There is new always-free offer on Oracle site:
– AMD Epyc CPU: 2 servers, each with: 1 core, 1 GB ram, 50 GB HDD, 480 mb/s connection
– Ampere ARM CPU: 4 servers, each with: 1 core 2.8GHz, 6 GB ram, 50 GB HDD, 1gb/s connection
– 10 TB/month transfer limit for all your machines together

These 4 Ampere servers you can combine into 1 server: 4×2.8GHz, 24 GB ram, 50 GB HDD with 4gb/s connection!
You can also create 6 VPSes for 6 small projects.

Registration:
https://www.oracle.com/pl/cloud/free/
You will have to add credit card to account. Virtual/prepaid cards are not accepted. Revolut virtual card generated in smartphone application is not accepted, but physical Revolut card is ok.

It takes around 15 minutes to activate account after you register.

How to order free 4 core VPS and unlock internet access

When you order any dedic or VPS, it always comes with all internet access opened by default. Here we get machine with almost everything closed (except port 22 for SSH). Every port is blocked in Oracle Cloud panel and in Ubuntu.
If you install nginx on it, it will say in webbrowser that site is offline. Read next 16 steps to unlock all ports.

Go to https://cloud.oracle.com/ and login to your account. Click on ‘Create a VM instance’.

Click on ‘Edit Image and shape’.

Click on ‘Change image’.

Select ‘Canonical Ubuntu’ and click ‘Select image’.

Click on ‘Change shape’.
1. Select shape ‘Ampere’
2. Unroll options
3. Change OCPUs number to 4, it will automatically change RAM to 24 GB
4. Tick ‘VM.Standard.A1.Flex’
5. Click ‘Select shape’

For next step machine configuration step, you must have SSH key pair. If you already have one, you can skip next 3 steps.

To generate key pair you need ‘Git Bash’. You can download it here: https://git-scm.com/download/win

Run Git Bash using Windows Search. Just type ‘git’, it should find it.

In Git Bash window type:

ssh-keygen.exe -t rsa

It will generate SSH key pair and save it in your Windows user directory, in subdirectory .ssh. This directory may be hidden (depends on Windows settings), so you can’t easily navigate there. In this case, you can open any directory in Windows explorer, click on address bar, type:

%HOMEPATH%/.ssh

and click Enter. It should open folder with your id_rsa.pub file. You can copy it on desktop or to Downloads folder, to make it easy to find.

On Oracle page scroll to ‘Add SSH Keys’ and select option ‘Upload public key files (.pub)’.

Select file ‘id_rsa.pub’ we generated in previous 3 steps in file picker (button ‘Browse’). Oracle will upload it to your new VPS. Git Bash with automatically login to your server using this key – using all keys which are in .ssh directory – when you use ‘ssh’ command in it.

Your machine should change status to ‘Running’ within 1 minute. On right top side of page, there will appear IP of your server. Copy it. We will use it later.

There will also appear ‘Virtual cloud network’ with some random name. Click on it.

Click on Subnet name – again some random value.

Click on Default Security List name – again some random value.

Click ‘Add Ingress Rules’.

In ‘source CIDR’ type:

0.0.0.0/0

and click ‘Add Ingress Rules’.

Network TCP – like Tibia and www – traffic is unlocked in Oracle Cloud panel. You may also unblock UDP protocol traffic, if you plan to host TeamSpeak or other application which uses UDP.

Now we got to unlock traffic in Ubuntu. Open Git Bash and type:

ssh ubuntu@x.x.x.x

with your server IP in place of x.x.x.x, like ssh ubuntu@138.3.243.101
On first connection it will ask you, if you are sure that you connected to valid server. Type ‘yes’ and press Enter.

You are now connected to your VPS, but you still can’t run web server or OTS on it. Type:

sudo su
iptables -I INPUT -j ACCEPT
iptables-save > /etc/iptables/rules.v4

It will unlock all incoming traffic in Ubuntu.

[OTS] Lists of unique numbers kept in ‘player storage’

Friend asked me about code to keep ‘auto loot lists’ in storage. So player can configure few lists of items and then just switch between them, when he goes to other spawn with different loot.

-- config
local listsStartStorage = 550000000
local maximumListsCount = 10
local maximumElementsPerList = 100

-- variables for calculations
local listOffset = maximumElementsPerList + 1

function lists_getLists(player)
    local result = {}
    for i = 1, maximumListsCount do
        result[i] = lists_getListElementsCount(player, i)
    end

    return result
end

function lists_getListElementsCount(player, listId)
    local startStorage = listsStartStorage + listOffset * listId
    return math.max(0, player:getStorage(startStorage))
end

function lists_getListElements(player, listId)
    local result = {}
    local startStorage = listsStartStorage + listOffset * listId
    local listElementsCount = lists_getListElementsCount(player, listId)
    for i = startStorage + 1, startStorage + listElementsCount do
        table.insert(result, player:getStorage(i))
    end

    return result
end

function lists_removeAllListElements(player, listId)
    local startStorage = listsStartStorage + listOffset * listId
    local listElementsCount = lists_getListElementsCount(player, listId)
    for i = startStorage + 1, startStorage + listElementsCount + 1 do
        player:setStorage(i, -1)
    end
    player:setStorage(startStorage, -1)
end

function lists_getListElement(player, listId, elementKey)
    local elementStorage = listsStartStorage + listOffset * listId + elementKey
    return player:getStorage(elementStorage)
end

local function lists_assertListId(listId)
    assert(listId >= 1, 'minimum list id is 1')
    assert(listId <= maximumListsCount, 'maximum list id is 1' .. maximumListsCount)
end

local function lists_assertElementKey(elementKey)
    assert(elementKey >= 1, 'minimum list element id is 1')
    assert(elementKey <= maximumElementsPerList, 'maximum list element id is ' .. maximumElementsPerList)
end

-- this is for internal use only, should not be used by user
local function lists_setListElement(player, listId, elementKey, elementValue)
    lists_assertListId(listId)
    lists_assertElementKey(elementKey)
    local elementStorage = listsStartStorage + listOffset * listId + elementKey
    player:setStorage(elementStorage, elementValue)
end

-- this is for internal use only, should not be used by user
local function lists_setListElementsCount(player, listId, count)
    lists_assertListId(listId)
    lists_assertElementKey(count)
    local startStorage = listsStartStorage + listOffset * listId
    player:setStorage(startStorage, count)
end

function lists_addListElement(player, listId, elementValue)
    local elements = lists_getListElements(player, listId)
    for k, v in pairs(elements) do
        if v == elementValue then
            return
        end
    end

    local elementsCount = lists_getListElementsCount(player, listId)
    lists_setListElement(player, listId, elementsCount + 1, elementValue)
    lists_setListElementsCount(player, listId, elementsCount + 1)
end

function lists_removeListElement(player, listId, elementValue)
    local elements = lists_getListElements(player, listId)
    local elementToRemovePosition = nil
    for k, v in pairs(elements) do
        if v == elementValue then
            elementToRemovePosition = k
            break
        end
    end

    if elementToRemovePosition then
        local elementsCount = lists_getListElementsCount(player, listId)
        for i = elementToRemovePosition + 1, elementsCount  do
            local nextElementValue = lists_getListElement(player, listId, i)
            lists_setListElement(player, listId, i - 1, nextElementValue)
        end
        lists_setListElement(player, listId, elementsCount, -1)
        lists_setListElementsCount(player, listId, elementsCount - 1)
    end
end

TESTS:

Player = {}
function Player:new()
    o = {storages = {}}
    setmetatable(o, self)
    self.__index = self
    return o
end

function Player.getStorage(self, k)
    if (self.storages[k]) then
        return self.storages[k]
    end
    return -1
end

function Player.setStorage(self, k, v)
    if (v == -1) then
        v = nil
    end
    self.storages[k] = v
end

function Player.dumpStorages(self)
    local storages = 'storages: '
    for k, v in pairs(self.storages) do
        storages = storages .. k .. '=' .. v .. ', '
    end
    print(storages)
end

local function printPlayerListsElementsCount(player)
    local playerLists = lists_getLists(player)
    for k, v in pairs(playerLists) do
        local listElementsString = ''
        for k2, v2 in pairs(lists_getListElements(player, k)) do
            listElementsString = listElementsString .. ' ' .. k2 .. '=' .. v2 .. ', '
        end
        print ('list', k, 'elements', v, 'values', listElementsString)
    end
end

player = Player:new()

print()
player:dumpStorages()
print('start - empty list')
printPlayerListsElementsCount(player)

print()
player:dumpStorages()
lists_addListElement(player, 2, 2001)
lists_addListElement(player, 2, 2002)
print('added 2 elements to list 2')
player:dumpStorages()
printPlayerListsElementsCount(player)

print()
player:dumpStorages()
lists_addListElement(player, 1, 1001)
lists_addListElement(player, 1, 1002)
lists_addListElement(player, 1, 1003)
lists_addListElement(player, 1, 1004)
lists_addListElement(player, 1, 1005)
print('added 5 elements to list 1')
player:dumpStorages()
printPlayerListsElementsCount(player)

print()
player:dumpStorages()
lists_removeListElement(player, 1, 1001)
print('removed first element from list 1')
player:dumpStorages()
printPlayerListsElementsCount(player)

print()
player:dumpStorages()
lists_removeListElement(player, 1, 1005)
print('removed last element from list 1')
player:dumpStorages()
printPlayerListsElementsCount(player)

print()
player:dumpStorages()
lists_removeListElement(player, 1, 1003)
print('removed middle element from list 1')
player:dumpStorages()
printPlayerListsElementsCount(player)

print()
player:dumpStorages()
lists_removeAllListElements(player, 1)
print('removed all elements from list 1')
player:dumpStorages()

You can paste code and tests to online tool like: https://www.tutorialspoint.com/execute_lua_online.php

Compile otservbr and TFS 1.3 on Ubuntu 20.04 and Debian 10/11

Dockerfiles tested on 2021-08-24. You can comment otservbr / forgottenserver part to compile just one sources.

Ubuntu 20.04

FROM ubuntu:20.04

RUN apt-get update

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Europe/London

RUN apt-get -y install tzdata

# 1: forgottenserver (TFS)
RUN apt-get -y install git cmake build-essential libluajit-5.1-dev zip ca-certificates pkg-config autoconf libmariadb-dev-compat libboost-date-time-dev libboost-filesystem-dev libboost-system-dev libboost-iostreams-dev libpugixml-dev libcrypto++-dev libfmt-dev

RUN cd /home/ && git clone --depth 1 https://github.com/otland/forgottenserver.git

RUN cd /home/forgottenserver/ && mkdir build && cd build && cmake ..
RUN cd /home/forgottenserver/build/ && make -j 16

# 2: otservbr-global
RUN apt-get install -y git cmake build-essential libluajit-5.1-dev zip ca-certificates curl zip unzip tar pkg-config yasm autoconf

RUN cd /home/ && git clone https://github.com/microsoft/vcpkg && cd vcpkg && ./bootstrap-vcpkg.sh
RUN cd /home/vcpkg/ && ./vcpkg --triplet x64-linux install boost-asio boost-filesystem boost-iostreams boost-lockfree boost-system boost-variant cryptopp curl jsoncpp libmariadb pugixml spdlog

RUN cd /home/ && git clone --depth 1 https://github.com/opentibiabr/otservbr-global.git && cd otservbr-global && git checkout develop
RUN cd /home/otservbr-global/ && mkdir build && cd build && cmake -DCMAKE_TOOLCHAIN_FILE=../../vcpkg/scripts/buildsystems/vcpkg.cmake ..
RUN cd /home/otservbr-global/build/ && make -j 16

Debian 11

FROM debian:11

RUN apt-get update

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Europe/London

RUN apt-get -y install tzdata

# 1: forgottenserver (TFS)
RUN apt-get -y install git cmake build-essential libluajit-5.1-dev zip wget ca-certificates pkg-config autoconf libmariadb-dev-compat libboost-date-time-dev libboost-filesystem-dev libboost-system-dev libboost-iostreams-dev libpugixml-dev libcrypto++-dev libfmt-dev

RUN apt remove -y libfmt-dev
RUN cd /root/ && wget https://github.com/fmtlib/fmt/releases/download/7.1.3/fmt-7.1.3.zip && unzip fmt-7.1.3.zip
RUN cd /root/fmt-7.1.3/ && mkdir build && cd build && cmake .. && make -j 16 && make install

RUN cd /home/ && git clone --depth 1 https://github.com/otland/forgottenserver.git

RUN cd /home/forgottenserver/ && mkdir build && cd build && cmake ..
RUN cd /home/forgottenserver/build/ && make -j 16

# 2: otservbr-global
RUN apt-get install -y git cmake build-essential libluajit-5.1-dev wget zip ca-certificates curl zip unzip tar pkg-config yasm autoconf

RUN apt remove -y cmake
RUN cd /root/ && wget https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1.tar.gz
RUN cd /root/ && tar -zxvf cmake-3.21.1.tar.gz && cd cmake-3.21.1 && ./bootstrap -- -DCMAKE_USE_OPENSSL=OFF && make -j 16 && make install

RUN cd /home/ && git clone https://github.com/microsoft/vcpkg && cd vcpkg && ./bootstrap-vcpkg.sh
RUN cd /home/vcpkg/ && ./vcpkg --triplet x64-linux install boost-asio boost-filesystem boost-iostreams boost-lockfree boost-system boost-variant cryptopp curl jsoncpp libmariadb pugixml spdlog

RUN cd /home/ && git clone --depth 1 https://github.com/opentibiabr/otservbr-global.git && cd otservbr-global && git checkout develop
RUN cd /home/otservbr-global/ && mkdir build && cd build && cmake -DCMAKE_TOOLCHAIN_FILE=../../vcpkg/scripts/buildsystems/vcpkg.cmake ..
RUN cd /home/otservbr-global/build/ && make -j 16

Debian 10

FROM debian:10

RUN apt-get update

ARG DEBIAN_FRONTEND=noninteractive
ENV TZ=Europe/London

RUN apt-get -y install tzdata

# 1: forgottenserver (TFS)
RUN apt-get -y install git cmake build-essential libluajit-5.1-dev zip wget ca-certificates pkg-config autoconf libmariadb-dev-compat libboost-date-time-dev libboost-filesystem-dev libboost-system-dev libboost-iostreams-dev libpugixml-dev libcrypto++-dev

RUN apt remove -y libfmt-dev
RUN cd /root/ && wget https://github.com/fmtlib/fmt/releases/download/7.1.3/fmt-7.1.3.zip && unzip fmt-7.1.3.zip
RUN cd /root/fmt-7.1.3/ && mkdir build && cd build && cmake .. && make -j 16 && make install

RUN cd /home/ && git clone --depth 1 https://github.com/otland/forgottenserver.git

RUN cd /home/forgottenserver/ && mkdir build && cd build && cmake ..
RUN cd /home/forgottenserver/build/ && make -j 16

# 2: otservbr-global
RUN apt-get install -y git cmake build-essential libluajit-5.1-dev wget zip ca-certificates curl zip unzip tar pkg-config yasm autoconf

RUN apt remove -y cmake
RUN cd /root/ && wget https://github.com/Kitware/CMake/releases/download/v3.21.1/cmake-3.21.1.tar.gz
RUN cd /root/ && tar -zxvf cmake-3.21.1.tar.gz && cd cmake-3.21.1 && ./bootstrap -- -DCMAKE_USE_OPENSSL=OFF && make -j 16 && make install

RUN cd /home/ && git clone https://github.com/microsoft/vcpkg && cd vcpkg && ./bootstrap-vcpkg.sh
RUN cd /home/vcpkg/ && ./vcpkg --triplet x64-linux install boost-asio boost-filesystem boost-iostreams boost-lockfree boost-system boost-variant cryptopp curl jsoncpp libmariadb pugixml spdlog

RUN cd /home/ && git clone --depth 1 https://github.com/opentibiabr/otservbr-global.git && cd otservbr-global && git checkout develop
RUN cd /home/otservbr-global/ && mkdir build && cd build && cmake -DCMAKE_TOOLCHAIN_FILE=../../vcpkg/scripts/buildsystems/vcpkg.cmake ..
RUN cd /home/otservbr-global/build/ && make -j 16

[OTS] Check monsters spawn time in spawn.xml

This script reads spawn.xml file and list monsters sorted by spawn time. You can easily detect monsters with too low or too high spawn times.

<?php

$spawnsFile = 'ots-spawn.xml';

function element_attributes($element_name, $xml)
{
    if ($xml == false) {
        return false;
    }
    $found = preg_match(
        '#<' . $element_name .
        '\s+([^>]+(?:"|\'))\s?/?>#',
        $xml,
        $matches
    );
    if ($found == 1) {
        $attribute_array = array();
        $attribute_string = $matches[1];
        $found = preg_match_all(
            '#([^\s=]+)\s*=\s*(\'[^<\']*\'|"[^<"]*")#',
            $attribute_string,
            $matches,
            PREG_SET_ORDER
        );
        if ($found != 0) {
            foreach ($matches as $attribute) {
                $attribute_array[$attribute[1]] =
                    substr($attribute[2], 1, -1);
            }
            return $attribute_array;
        }
    }
    return false;
}

$monsters = [];

foreach (file($spawnsFile) as $lineNo => $line) {
    $line = trim($line);
    if (substr($line, 0, 8) === '<monster') {
        $monsterAttributes = element_attributes('monster', $line);
        if (is_array($monsterAttributes)) {
            if (isset($monsterAttributes['name']) && isset($monsterAttributes['spawntime'])) {
                $name = $monsterAttributes['name'];
                $spawnTime = $monsterAttributes['spawntime'];

                $monsters[] = [
                    'line' => $lineNo,
                    'name' => $monsterAttributes['name'],
                    'spawnTime' => (int)$monsterAttributes['spawntime'],
                ];
            }
        }
    }
}

usort($monsters, function ($a, $b) {
    return $a['spawnTime'] < $b['spawnTime'];
});

foreach ($monsters as $monster) {
    echo $monster['line'] . ',' . $monster['name'] . ',' . $monster['spawnTime'] . PHP_EOL;
}

PHP file, directory, soft link and hard link information

What PHP functions file_exists, is_file, is_dir and is_link return for file, directory, hard link and soft link?

<?php

/*
touch file
mkdir dir
ln file file_hard_link
ln -s file file_soft_link
ln -s dir dir_soft_link
touch dir/file_in_dir
echo 1 > file
cat file file_hard_link file_soft_link
*/

$tests = [
    'file',
    'dir',
    'file_hard_link',
    'file_soft_link',
    'dir_soft_link',
    'dir/file_in_dir',
    'dir_soft_link/file_in_dir'
];

foreach ($tests as $test) {
    echo 'name: ', str_pad($test, 25) . ' ';
    echo 'file_exists: ', (int)file_exists($test) . ' ';
    echo 'is_file: ', (int)is_file($test)  . ' ';
    echo 'is_dir: ', (int)is_dir($test) . ' ';
    echo 'is_link: ', (int)is_link($test) . PHP_EOL;
}

Result:

name: file                      file_exists: 1 is_file: 1 is_dir: 0 is_link: 0
name: dir                       file_exists: 1 is_file: 0 is_dir: 1 is_link: 0
name: file_hard_link            file_exists: 1 is_file: 1 is_dir: 0 is_link: 0
name: file_soft_link            file_exists: 1 is_file: 1 is_dir: 0 is_link: 1
name: dir_soft_link             file_exists: 1 is_file: 0 is_dir: 1 is_link: 1
name: dir/file_in_dir           file_exists: 1 is_file: 1 is_dir: 0 is_link: 0
name: dir_soft_link/file_in_dir file_exists: 1 is_file: 1 is_dir: 0 is_link: 0

[OTClient] Generate full minimap for your server

Example for Tibia version 8.54

TO GENERATE MINIMAP FOR BIG MAPS – LIKE RL TIBIA – YOU NEED TO COMPILE 64-bit VERSION OF OTCLIENT!
Loading big map requires few GB RAM. 32-bit (x86) version of OTClient can load maps up to ~40MB OTBM. Using it with bigger map will result in not loading whole map -> there will be places missing on minimap!

Put client Tibia.spr and Tibia.dat files in data/things/854/

Put server files items.otb, world.otbm (map) and map .xml files in data/things/854/

Run OTClient and open Terminal (CTRL+T in client). Type:

g_game.setClientVersion(854)
g_things.loadOtb('things/854/items.otb')
g_map.loadOtbm('things/854/world.otbm')
g_minimap.saveOtmm('minimap.otmm')

If you are using OTClient_mapgen v4 ( https://github.com/gesior/otclient_mapgen/releases ):

prepareClient(854, 'things/854/items.otb', 'things/854/world.otbm', 1, 1)
saveMinimap('minimap.otmm')

Close client.

Your minimap file will be in folder:

  • Windows: %HOMEPATH%/otclient (paste it as path to folder in Windows Explorer)
  • Linux: $HOME/.otclient (cd to it)

Copy it to data folder of OTClient. OTClient searches for files first in this folder. That way you can share it with players as part of your client.