What is UTF-8? UTF-8 is an encoding standard that can represent all Unicode characters. This allows to show almost any writing system in the world.
What's new in SMF 1.1 RC3 with regard to UTF-8? SMF has always supported multiple character sets. Each language package was written in a specific character set. Nothing has changed to the support of those character sets, but additional support for UTF-8 has been added. It is possible to convert your forum to UTF-8 or (in case of a new forum) install with UTF-8 support. If you have your forum in UTF-8 mode, both the database and website will be using UTF-8.
The following character sets are currently used for SMF's language packages (both 1.0.x and 1.1):
| Character set | Language |
| big5 | Chinese (traditional) |
| gbk | Chinese (simplified) |
| ISO-8859-1 | Albanian, Brazilian, Catalan, Danish, Dutch, English, Finnish, French, German, Portuguese, Norwegian, Spanish, Swedish |
| ISO-8859-2 | Croatian, Hungarian, Polish, Romanian |
| ISO-8859-9 | Turkish |
| tis-620 | Thai |
| UTF-8 | Chinese (simplified), Chinese (traditional), Japanese, Persian |
| windows-1256 | Arabic |
| windows-1251 | Bulgarian, Russian |
| windows-1253 | Greek |
| windows-1255 | Hebrew |
As of SMF 1.1 RC3 you'll be able to also download each of those language packages in UTF-8 character set ([" class="bbc_link">Download] ->[" class="bbc_link">Language packs]).
Why would I need UTF-8?
There are a few reasons you might need UTF-8:
- If you want to support multiple languages that use different character sets on your forum. For instance if you want to support both Russian and Turkish, you will need a character set that supports both. UTF-8 is then a logical choice.
- If the software integrating with SMF uses UTF-8. In some cases such an integration can require character sets to match.
- If you need better search results or improved sorting. In some cases searching and sorting by the database can be improved by chosing UTF-8 as your character set.
Why would I NOT need UTF-8? For all the other reasons, UTF-8 would probably not be very useful. Besides, it's a bit slower too.
Also keep in mind that you need at least MySQL 4.1 and SMF 1.1 RC3 to be able to use UTF-8 as default character set if you are using MySQL as your database scheme.
How to convert to UTF-8?
- Start with a backup of your database(!) Character set conversions are all but guaranteed to go right.
- Go to 'Forum Maintenance' -> 'Convert the database and data to UTF-8'
- Select the character set your current data is in. The default setting for this is based on the character set of your default langauge file.
- After pressing proceed, your database will be converted. Depending on the size of your database, the conversion process might stop temporarily from time to time to avoid overloading the server. If that was successful, your forum should be converted to UTF-8.
- You'll be needing a new set of language files. All language files need to be UTF-8 compatible. Luckily all language packs for 1.1 RC3 are available for both the original character set and UTF-8, so simply download them and you should be ready to go.
- Once all the UTF-8 language packs have been installed, convert the language settings of each user by running the following query:
UPDATE smf_members
SET lngfile = CONCAT(lngfile, '-utf8')
WHERE lngfile != ''
- Also, change the default language in your admin center - Admin -> Server Settings
- Check to see if all your data was properly converted
- If any of your posts contain HTML entities, you will want to convert those to UTF-8 as well...
