From Windows 10 to OTS development machine for dummies

I made this tutorial on clean machine just after installing Windows 10 Pro (64bit) on 2020-02-16.
Installing everything took around 1 hour.
In this tutorial we will use Visual Studio 2019, not 2017 recommended for TFS compilation. It will work!

What do you need before start?

  • Windows 10
  • Web Browser (tutorial created with Google Chrome)

What will you have after completing this tutorial?

  • GIT client integrated with Windows 10
  • ‘linux console’ integrated with Windows 10
  • Visual Studio 2019 Community with C++ dev tools – IDE that let you compile TFS/OTClient easily (generate ‘.exe’ file)
  • vcpkg – C++ Library Manager for Windows, it will let you compile TFS/OTClient without searching for C++ libraries for hours
  • IDEA Community Edition with EmmyLUA plugin – free version of IDE that let you manage ‘data’ folder easily (XML and LUA files)
  • MariaDB database, PHP 7.4 and Apache2 server – website for your OTS
  • The Forgotten Server 1.2+ running on your PC
  • Gesior2012 for TFS 1.2+ running on your PC

1. Install GIT for Windows with environment integration

Download it from site (version 64-bit Setup): https://git-scm.com/download/win
‘Setup’ version will integrate Git with Windows explorer.

Download 64-bit Git for Windows Setup.
Unselect GUI option. It’s useless. You will have better Git GUI in IDEA.
Change line ending to ‘as is’. On GitHub most of code uses Unix-style endings.

2. Install Visual Studio 2019 Community with C++

Download it from site: https://visualstudio.microsoft.com/vs/

Select “Community” version. It will start download. After download run installer.
Select packet “Programming classic applications in C++”.
Select packet “English language”. It’s required to install TFS compilation libraries.
After installation run Visual Studio for first time to make it configure itself.
You don’t need Microsoft Account to use it. Click ‘Not now, maybe later’.

3. Install vcpkg – C++ library manager for Windows

Open Git Bash (‘linux console’) in folder ‘C:\’
Type in console: git clone https://github.com/Microsoft/vcpkg.git
git clone https://github.com/Microsoft/vcpkg.git
Type in console: cd vcpkg
cd vcpkg
Open website: https://github.com/Microsoft/vcpkg and find newest tagged version

Link: https://github.com/Microsoft/vcpkg
Newest tagged version should be stable. I tried master branch and it did not work with all libraries. You can always use old version, not newest. I used for this tutorial: 2020.01

Type in console: git checkout 2020.01
git checkout 2020.01

It will switch version of vcpkg to version from 2020.01

Type: ./bootstrap-vcpkg.bat
./bootstrap-vcpkg.bat

To build vcpkg. You must do it after downloading from GitHub and after every version change.

Run Windows PowerShell as Administrator (another black console..)
Change path in console to vcpkg path
Type: cd C:\vcpkg
cd C:\vcpkg
In console type: .\vcpkg integrate install
.\vcpkg integrate install

This will integrate vcpkg C++ library manager with Visual Studio we installed before.
After integrating with Visual Studio close PowerShell. We don’t need administrator rights anymore.

If you closed Git Bash console we opened at start, open it again in vcpkg folder.
Copy current libraries list from official TFS wiki (for 64-bit): https://github.com/otland/forgottenserver/wiki/Compiling-on-Windows-%28vcpkg%29

Libraries list at 2020-02-17:

vcpkg install boost-iostreams:x64-windows boost-asio:x64-windows boost-system:x64-windows boost-filesystem:x64-windows boost-variant:x64-windows boost-lockfree:x64-windows luajit:x64-windows libmariadb:x64-windows pugixml:x64-windows mpir:x64-windows cryptopp:x64-windows
Type in console: ./ and paste list

./ means ‘run’

We got all libraries for TFS!

4. Download TFS and compile engine

Create folder:

C:\ots
Open Git Bash in folder C:\ots
Download newest TFS. Type: git clone https://github.com/otland/forgottenserver.git
git clone https://github.com/otland/forgottenserver.git
C:\ots\forgottenserver\vc14\theforgottenserver.sln

It will open TFS project in Visual Studio.

Click ‘OK’ when VS ask you about changing target platform.

Change build type to Release and build target platform to x64.
Previously we installed C++ libraries for 64-bit platform – we can compile only 64-bit version.

If you plan to do many changes in sources and compile in often, you may try to use Debug build type.
It should link faster and reduce time of building solution.

Start compilation of TFS. It can take few minutes.
Open folder: C:\ots\forgottenserver\vc14\x64\Release
C:\ots\forgottenserver\vc14\x64\Release
Copy all .dll files and .exe file to folder: C:\ots\forgottenserver\
C:\ots\forgottenserver\
TFS is ready. Now we need to install database server and import TFS database schema.

5. Install MariaDB, PHP and Apache2 servers

Download newest XAMPP for Windows.

Link: https://www.apachefriends.org/download.html

During installation deselect components we won’t use for OTS development.
Start Apache, MySQL and open phpMyAdmin site (database administration site).

If Apache fails to start, it’s probably because some other application is already using port 80.
Common problem is Skype. Turn off Skype, start Apache, start again Skype – you got website and Skype working.

In phpMyAdmin open page with new user account configuration.
Type username forgottenserver
Type secure password for that user
Check ‘Create database with same name and grant all access.’
New database will appear in left panel. Click it and go to Import page.
Choose file: C:\ots\forgottenserver\schema.sql
C:\ots\forgottenserver\schema.sql

and click Import.
We got database for our OTS! Time to configure game server.

Open folder C:\ots\forgottenserver and copy file config.lua.dist to config.lua

6. Install IntelliJ IDEA, configure game server and run it for first time!

What is IntelliJ IDEA? It’s IDE for Java and Android development.
What is IDE? It’s ‘Integrated Development Environment’.
Why do I need it? To develop code faster, automatically detect simple bugs and make cleaner code.

Choose folder: C:\ots\forgottenserver
C:\ots\forgottenserver
Open IntelliJ IDEA Settings
Go to Plugins and install EmmyLua
Restart IDE to make it work with LUA
Configure your password to database user forgottenserver in file config.lua
Your server is running!

7. Install website (Gesior2012)

Open folder: C:\xampp\htdocs
 C:\xampp\htdocs
Remove all files from C:\xampp\htdocs
Open Git Bash in folder C:\xampp\htdocs
Type: git clone https://github.com/gesior/Gesior2012.git .
git clone https://github.com/gesior/Gesior2012.git .

Yes. There is space and dot at end. We want to download Gesior2012 to current folder. Not to folder C:\xampp\htdocs\Gesior2012

Type: git checkout TFS-1.2
git checkout TFS-1.2

Change Gesior2012 version to TFS-1.2 (exactly change git branch to TFS-1.2)
Same as we did during vcpkg installation. Download from GitHub with git clone and then change version.

From now you will edit every directory in IntelliJ IDEA 🙂

PHP files are not colored in IntelliJ IDEA Community edition 🙁
LUA plugin is free, but PHP is available only in IntelliJ IDEA Ultimate (paid / 30 days trial) or as separate application PHPStorm (it’s IntelliJ IDEA with PHP plugin).

Open website http://127.0.0.1 and click link to installation.
Refresh site. Error about IP should disappear.
There will be probably some errors about cache folder.
We got to set cache folder not read-only.
When it asks about ‘To all files and sub-directories’ click Yes.
Click link to Step 1 in left side menu.
Type path to TFS: C:\ots\forgottenserver
C:\ots\forgottenserver

Continue steps 2-5.

Your website should be ready to use!

Reset TFS database without removing accounts, players and guilds

SQL queries to reset database to ‘server start’ without removing accounts, players and guilds. All players are set to level 30 with valid hp, mana and cap for their vocations.

UPDATE `players`
SET
`health` = 185 + 22 * 5,
`healthmax` = 185 + 22 * 5,
`mana` =  35 + 22 * 30,
`manamax` = 35 + 22 * 30,
`cap` = 470 + 22 * 10,
`level` = 30,
`experience` = 368300,
`maglevel` = 30,
`soul` = 100,
`manaspent` = 0,
`lookaddons` = 0,
`posx` = 0,
`posy` = 0,
`posz` = 0, 
`conditions` = '',
`skull` = 0,
`skulltime` = 0,
`blessings` = 0,
`offlinetraining_time` = 43200,
`offlinetraining_skill` = -1,
`stamina` = 2520,
`skill_fist` = 10,
`skill_club` = 10,
`skill_sword` = 10,
`skill_axe` = 10,
`skill_dist` = 10,
`skill_shielding` = 10,
`skill_fishing` = 10,
`skill_fist_tries` = 0,
`skill_club_tries` = 0,
`skill_sword_tries` = 0,
`skill_axe_tries` = 0,
`skill_dist_tries` = 0,
`skill_shielding_tries` = 0,
`skill_fishing_tries` = 0
WHERE `vocation` =  1 OR `vocation`= 5  OR `vocation`= 2 OR `vocation`= 6;

UPDATE `players`
SET
`health` = 185 + 22 * 10,
`healthmax` = 185 + 22 * 10,
`mana` =  35 + 22 * 15,
`manamax` = 35 + 22 * 15,
`cap` = 470 + 22 * 20,
`level` = 30,
`experience` = 368300,
`maglevel` = 5,
`soul` = 100,
`manaspent` = 0,
`lookaddons` = 0,
`posx` = 0,
`posy` = 0,
`posz` = 0, 
`conditions` = '',
`skull` = 0,
`skulltime` = 0,
`blessings` = 0,
`offlinetraining_time` = 43200,
`offlinetraining_skill` = -1,
`stamina` = 2520,
`skill_fist` = 10,
`skill_club` = 10,
`skill_sword` = 10,
`skill_axe` = 10,
`skill_dist` = 30,
`skill_shielding` = 10,
`skill_fishing` = 10,
`skill_fist_tries` = 0,
`skill_club_tries` = 0,
`skill_sword_tries` = 0,
`skill_axe_tries` = 0,
`skill_dist_tries` = 0,
`skill_shielding_tries` = 0,
`skill_fishing_tries` = 0
WHERE `vocation` =  3 OR `vocation`= 7;

UPDATE `players`
SET
`health` = 185 + 22 * 15,
`healthmax` = 185 + 22 * 15,
`mana` =  35 + 22 * 5,
`manamax` = 35 + 22 * 5,
`cap` = 470 + 22 * 25,
`level` = 30,
`experience` = 368300,
`maglevel` = 2,
`soul` = 100,
`manaspent` = 0,
`lookaddons` = 0,
`posx` = 0,
`posy` = 0,
`posz` = 0, 
`conditions` = '',
`skull` = 0,
`skulltime` = 0,
`blessings` = 0,
`offlinetraining_time` = 43200,
`offlinetraining_skill` = -1,
`stamina` = 2520,
`skill_fist` = 10,
`skill_club` = 30,
`skill_sword` = 30,
`skill_axe` = 30,
`skill_dist` = 10,
`skill_shielding` = 10,
`skill_fishing` = 10,
`skill_fist_tries` = 0,
`skill_club_tries` = 0,
`skill_sword_tries` = 0,
`skill_axe_tries` = 0,
`skill_dist_tries` = 0,
`skill_shielding_tries` = 0,
`skill_fishing_tries` = 0
WHERE `vocation` =  4 OR `vocation`= 8;

DELETE FROM `account_ban_history`;
DELETE FROM `account_bans`;
DELETE FROM `house_lists`;
DELETE FROM `houses`;
DELETE FROM `market_history`;
DELETE FROM `market_offers`;
DELETE FROM `player_deaths`;
DELETE FROM `player_depotitems`;
DELETE FROM `player_inboxitems`;
DELETE FROM `player_items`;
DELETE FROM `player_spells`;
DELETE FROM `player_storage`;
DELETE FROM `players_online`;
DELETE FROM `tile_store`;

items.xml sorter – by id/fromid

Simple PHP code to sort file items.xml by id/fromid attribute

<?php
$path_to_file = 'data/items/items.xml';
$xml = simplexml_load_string(file_get_contents($path_to_file));
$trees = $xml->xpath('/items/item');

function sort_trees($t1, $t2)
{
    $a = isset($t1['id']) ? $t1['id'] : $t1['fromid'];
    $b = isset($t2['id']) ? $t2['id'] : $t2['fromid'];
    return $a - $b;
}

usort($trees, 'sort_trees');

foreach ($trees as $tree) {
    echo '<item ';
    foreach ($tree->attributes() as $k => $v) {
        echo $k . '="' . $v . '" ';
    }
    if (count($tree->attribute) > 0) {
        echo '>' . PHP_EOL;
        foreach ($tree->attribute as $k => $v) {
            echo '<attribute key="' . $v->attributes()['key'] . '" value="' . htmlspecialchars($v->attributes()['value'], ENT_XML1 | ENT_QUOTES, 'UTF-8') . '" />' . PHP_EOL;
        }
        echo '</item>' . PHP_EOL;
    } else {
        echo '/>' . PHP_EOL;
    }
}

[TFS 1.x+] Generate big depot

LUA code to generate backpack full of backpacks:

function fillContainer(container, level, innerContainerCount)
    if level > 0 then
        for i = 1, innerContainerCount do
            local innerContainer = Game.createItem(2000)
            container:addItemEx(innerContainer)
            fillContainer(innerContainer, level - 1, innerContainerCount)
        end
    else
        for i = 1, 20 do
            local item = Game.createItem(i % 2 == 0 and 2376 or 2400)
            container:addItemEx(item)
        end
    end
end

function generateBigContainer(player, innerContainerCount, deepness)
    local item = player:getTile():addItem(2000)
    fillContainer(item, deepness, innerContainerCount)
    return item
end

Code to call it:

!lua generateBigContainer(player, 20, 2)

Requires !lua debug:

https://otland.net/threads/debug-lua-scripts-with-talkaction.112027/

https://otland.net/threads/tfs-1-x-luajit-debug-lua-scripts-with-talkaction.260910/

https://otland.net/threads/tfs-1-3-lua-5-2-debug-lua-scripts-with-talkaction.266086/

PHP TibiaCam file reader

Simple reader for uncompressed TibiaCam files (binary files, up to 60MB).

It’s also example of fast reading binary data from file and handling it as stream.

<?php

class BinaryData
{
    /** @var resource */
    private $data;
    /** @var int */
    private $length;

    /**
     * BinaryData constructor.
     * @param string $data
     */
    public function __construct($data)
    {
        $this->data = fopen('php://memory', 'wb+');
        fwrite($this->data, $data);
        $this->length = ftell($this->data);
        rewind($this->data);
    }

    /**
     * @return int
     */
    public function getLength()
    {
        return $this->length;
    }

    /**
     * @return bool
     */
    public function isEof()
    {
        return ftell($this->data) == $this->length;
    }

    /**
     * @param int $length
     */
    public function skip($length)
    {
        fseek($this->data, $length, SEEK_CUR);
    }

    /**
     * @return int
     */
    public function getU8()
    {
        return unpack('C', fread($this->data, 1))[1];
    }

    /**
     * @return int
     */
    public function getU16()
    {
        return unpack('v', fread($this->data, 2))[1];
    }

    /**
     * @return int
     */
    public function getU32()
    {
        return unpack('V', fread($this->data, 4))[1];
    }

    /**
     * @return int
     */
    public function getU64()
    {
        return unpack('P', fread($this->data, 8))[1];
    }

    /**
     * @param int $length
     * @return string
     */
    public function getString($length = -1)
    {
        if ($length == -1) {
            $length = $this->getU16();
        }

        return fread($this->data, $length);
    }

}

class NetworkPacket extends BinaryData
{
    /**
     * 0x01 = packets sent to player
     * 0x03 = packets received from player
     * 0x05 = refresh packets for fast cam rewinding
     */
    const TYPE_OUT = 0x01;
    const TYPE_IN = 0x03;
    const TYPE_REFRESH = 0x05;

    /** @var int */
    private $type;

    /**
     * NetworkPacket constructor.
     * @param string $data
     */
    public function __construct($data)
    {
        parent::__construct($data);
        $this->type = $this->getU8();
    }

    /**
     * @return int
     */
    public function getType()
    {
        return $this->type;
    }

}

class Cam extends BinaryData
{
    /**
     * Cam constructor.
     * @param string $filePath path to cam file
     */
    public function __construct($filePath)
    {
        parent::__construct(file_get_contents($filePath));
    }

    /**
     * @return array
     */
    public function readTibiaCamAttributes()
    {
        $this->skip(8); // text: TIBIACAM

        $camVersion = $this->getU8();
        $playerId = $this->getU32();
        $playerName = $this->getString();
        $protocolVersion = $this->getU16();
        $timeCreated = $this->getU64();

        return [$camVersion, $playerId, $playerName, $protocolVersion, $timeCreated];
    }

}

$fileName = 'nowy.cam.ready';
$cam = new Cam($fileName);
$camAttributes = $cam->readTibiaCamAttributes();
var_dump($camAttributes);

while (!$cam->isEof()) {
    $packetTime = $cam->getU64();
    var_dump($packetTime);
    $packet = new NetworkPacket($cam->getString());
    var_dump($packet->getType());
    var_dump($packet->getLength());
}

OTS Restarter script

restart.sh with console logs with timestamp and binary file copy (for GDB)

#!/bin/bash
ulimit -c unlimited
while true; do

 cp kasteria bins/kasteria_`date '+%Y-%m-%d_%H-%M'`_`crc32 kasteria`
 stdbuf -o 0 ./kasteria 2>&1 | ts '%Y-%m-%d_%H-%M-%.S' | tee -a 'console/console_'`date '+%Y-%m-%d'`'.log';

 echo START SLEEP FOR 3 SECONDS, PRESS CTRL+C TO TURN OFF RESTARTER
 sleep 3
 echo END SLEEP

done

Autowalk/cavebot for OTClient

Someone asked me about script that will make character run to given position and after reaching it run to another. I made it for free, so I decided to publish it as it may be interesting for others.

It’s not ready-to-run module, but I think it’s very close. For tests I pasted parts of it to ‘battle’ module and it worked.

This code requires ‘BOT_PROTECTION’ disabled in OTClient.

local walkEvent = nil
local positionsList = {
    { x = 32608, y = 32131, z = 7 },
    { x = 32601, y = 32131, z = 7 },
    { x = 32601, y = 32138, z = 7 },
    { x = 32608, y = 32138, z = 7 },
}

local isAttacking = 0
local isFollowing = 0
local currentTargetPositionId = 1

local autowalkTargetPosition = positionsList[currentTargetPositionId]

function init()
    connect(LocalPlayer, {
        onPositionChange = onCreaturePositionChange
    })

    connect(g_game, {
        onAttackingCreatureChange = onAttack,
        onFollowingCreatureChange = onFollow,
        onDisappear = onCreatureDisappear
    })
end

function terminate()
    disconnect(LocalPlayer, {
        onPositionChange = onCreaturePositionChange
    })

    disconnect(g_game, {
        onAttackingCreatureChange = onAttack,
        onFollowingCreatureChange = onFollow,
        onDisappear = onCreatureDisappear
    })
end

function walkToTarget()
    if not g_game.isOnline() then
        walkEvent = scheduleEvent(walkToTarget, 500)
        return
    end

    if g_game.getLocalPlayer():getStepTicksLeft() > 0 then
        -- wait until walk animation finish
        walkEvent = scheduleEvent(walkToTarget, g_game.getLocalPlayer():getStepTicksLeft())
        return
    end

    if isAttacking or isFollowing or not autowalkTargetPosition then
        -- do not walk with selected attack/follow target or when there is no target position
        walkEvent = scheduleEvent(walkToTarget, 100)
        return
    end

    -- fast search path on minimap (known tiles)
    steps, result = g_map.findPath(g_game.getLocalPlayer():getPosition(), autowalkTargetPosition, 5000, 0)
    if result == PathFindResults.Ok then
        g_game.walk(steps[1], true)
    elseif result == PathFindResults.Position then
        -- on target position
        currentTargetPositionId = currentTargetPositionId + 1
        autowalkTargetPosition = positionsList[(currentTargetPositionId % #positionsList) + 1]
    else
        -- slow search path on minimap, if not found, start 'scanning' map
        steps, result = g_map.findPath(g_game.getLocalPlayer():getPosition(), autowalkTargetPosition, 25000, 1)
        if result == PathFindResults.Ok then
            g_game.walk(steps[1], true)
        end
    end

    -- limit steps to 10 per second (100 ms between steps)
    walkEvent = scheduleEvent(walkToTarget, math.max(100, g_game.getLocalPlayer():getStepTicksLeft()))
end

function onCreaturePositionChange(creature, newPos, oldPos)
    if not walkEvent then
        -- start automoving 1 second after logging into game
        walkEvent = scheduleEvent(walkToTarget, 1000)
    end
end

function onAttack(creature)
    isAttacking = creature and creature:getId() or 0
end

function onFollow(creature)
    isFollowing = creature and creature:getId() or 0
end

function onCreatureDisappear(creature)
    if isAttacking == creature:getId() then
        isAttacking = 0
    end
    if isFollowing == creature:getId() then
        isFollowing = 0
    end
end

PayPal (is a) scam – removing my PayPal account

PayPal wants to take fee for not realized transactions?! No more PayPal for me.

Tommorow I will remove my PayPal account and link all subscriptions directly to Visa – Gesior.pl

https://www.paypal.com/us/webapps/mpp/ua/upcoming-policies-full

Amendments to the PayPal Account User Agreement

We’re removing the flat rate pricing for sending money to friends and family members who have PayPal accounts in a country other than the United States and introducing a new variable fee of 5% based on the amount you send with a minimum of $0.99 and a maximum of $4.99 per transaction. We’re also removing any variation depending on the recipient’s country. 

We are changing the currency conversion spread to 3.25% over a base exchange rate in situations where you are a sender of money in a PayPal transaction.

We’re changing how we treat refunds. If you refund (partially or fully) a transaction to a buyer or a donation to a donor, there are no fees to make the refund, but the fees you originally paid as the seller will not be returned to you.

Eloth start scripts for GMs

ROOK

Create fast monster spawn

Spawn “Snake” every second (1000 ms) for 5 seconds:

!lua function fms(m, p, c) doSummonCreature(m, p, true, true) if (c > 1) then addEvent(fms, 1000, m, p, c-1) end end fms("Spider", getCreaturePosition(cid), 5)

Mass spawn around GOD:

!lua for i = 1, 20 do doSummonCreature("Spider", getCreaturePosition(cid), true, true) end

Spawn temple NPC

For FACC towns:

/s The Oracle

For PACC towns:

/s The Gatekeeper

MAIN

Open Desert Quest

Create teleports that allow 20+ levels do quest solo without items:

!lua doCreateTeleport(1387, Position(32672, 32102, 8), Position(32647, 32092, 7)) doCreateTeleport(1387, Position(32672, 32070, 8), Position(32673, 32089, 8))

Generate big depot

LUA code to generate backpack full of backpacks:

function fill(container, level)
	for i = 1, 20 do
		local innerContainer = doAddContainerItem(container, 2000)
		if level > 0 then
			fill(innerContainer, level - 1)
		end
	end
end
container = doCreateItem(2000, getThingPos(cid)) fill(container, 2)

One-liner for !lua debug talkaction:

function fill(c, l) for i = 1, 20 do ic = doAddContainerItem(c, 2000) if l > 0 then fill(ic, l -1) end end end c = doCreateItem(2000, getThingPos(cid)) fill(c, 2)

Then put it in parcel and send to depot.