A so-called test installation is a local installation of SMF or any other software package, right on your own computer, which is inaccessible for other users on the internet. You can compare it with a web server which only you can access. With it you can test modifications done to the forum before applying them on your production site.
Software you need
Depending on your platform, you need to install a software package.
WebMatrix
Microsoft has made an IDE available for Windows with which you can install various software installations with a few clicks.
AMP packages
A list of all AMP (Apache-MySQL-PHP) packages, required to run SMF and other softwares, can be found here: Wikipedia: List of AMP packages
Some of these packages include phpMyAdmin or a similar tool for database management which you may need. If not, you can install phpMyAdmin yourself from here, or a smaller alternative, Adminer, from here.
Installing SMF
After you have installed the program appropriate for your system, you can proceed and install SMF on your computer. You do not need to use FTP to transfer the files; just open the directory where the program is installed, and you will usually find a directory named "www" or "htdocs" in it, to which you can extract the archive.