Saturday, October 22, 2016

The Famicom Disk System

Undated June 2020

The Famicom Disk System was a first-party peripheral released for the Nintendo Famicom which would let Japanese consumers plays their Famicom games on propriety magnetic disks instead of chip-based cartridges. This article explores why such technology was developed and retailed, as well as why it never saw a release outside of the Japanese market. Despite the obscure history of the FDS to us westerners, it was released in Japan in 1986 where the peripheral had over 50% market penetration and where Famicom disk kiosks were not fully discontinued until 2003. Nintendo continued to service disk systems until October 2007. By the time I had finished this article and my research on this topic, I ended up with more questions than I had answers to in regards to this peripheral.

In the nineties Japanese Nintendo console peripherals were borderline urban legends in the US. The NES, SNES, and N64 all had add-on peripherals that were released in Japan but never in the US. I am under the impression that even a test release of a NES modem in Minnesota never actually saw the light of day for two reasons. First, with a supposed test release of 10,000 units one would expect to see units turn up for sale every once in a while on the Internet, or show up at swap meets; even if they were to be recalled once the market test was over. Second, all images of the modem look like they were taken from a promotional VHS tape, leading me to believe the modem was never even released. This sort of muddled history has led to a lot of fascination, speculation, and rumors about what all those expansion slots under our Nintendo consoles were for!

For the record, the Famicom Disk System plugs into the console port of the system, there is no expansion port on the bottom of a Famicom. However, the Famicom DOES has a 15-pin port on the front edge of the console. Because the console's two default controllers were not removable like the NES, peripheral devices had to be attached through this expansion port, rather than through a controller port as on the NES. This was commonly used for third party controllers, usually as a substitute for the built-in controllers, but sometimes also as a 3rd and 4th player.

The biggest contributor to the release of the FDS might have been the chip shortage of 1988. In 1986 there was a semiconductor trade pact between the US and Japan. While the pact was aimed at helping U.S. chip makers compete with Japan, at the insistence of the US (I was not able to find WHY Japan accepted this deal), Japan agreed to cut back on worldwide chip exportation and also gave up a twenty percent portion of their worldwide market share. The US government anticipated that this would give US chip makers a chance to grab a bigger piece of worldwide chip production, but instead American industries were unable to fulfill the twenty percent void left by the new agreement. The plan was an economic failure, and anticipating the political situation, the release of the FDS may have been a move by Nintendo to continue to produce games in Japan through the shortage. Disks were cheap, re-writable, inexpensive especially compared to the price of chips, and were in ready supply.

Early on, the chip shortage also lead me into a contradiction while writing this article. Nintendo may have simply played up the chip shortage to their advantage in order to sell a new hardware peripheral to consumers. In US markets, the release of Super Mario Bro. 2 and Zelda II: The Adventure of Link coincided with the chip shortage. A limited number of the two games were available during the '88 holiday season, and as a result, it created a bit of a media buzz. Even today, Nintendo is renowned for under supplying retailers in their orders. Besides this being acknowledged as a business practice in the 2014 book Console Wars, but go ahead and ask the mom and pop retailer in your town about their Nintendo orders with their distributor. Even at the time of this article, collectors are scrambling to find the NES Mini. And remember the Amiibo shortage before that? This practice of under supplying goes all the way back to the NES and Famicom.While the chip shortage certainly was real, Nintendo used it as leverage to create demand, and hype around their intellectual property while simultaneously avoiding flooding the market with too much product; a lesson learned from Atari and the video game market crash in 1983.

The diskettes themselves allowed for 128KB of storage, 64KB on each side, which was twice of the typical cartridge at the time. As the diskettes were re-writable, it was possible for the first time ever to actually save your home console games without a password system. Battery backed memory did not exist until the 1987 U.S. release of The Legend of Zelda. Before that, ASCII, the creators of the Wizardry series, actually made an accessory called The Turbofile which was a hard drive which plugged into the Famicom's controller expansion port, and could save games and characters' progress to. It also allowed players to import their characters into sequels of the Wizardy games.

The FDS also came with a RAM expansion cart which also included an extra sound channel and a second graphics processor for scaling, rotation, and compression; which only a few un-noteworthy games used unfortunately. The second graphics processor was technically impressive for the Famicom, but in the end was not implemented in any standout titles. Despite the RAM adapter being advertised as a memory expansion to consumers, that was a bit misleading. Game data would be read from the disk just like a floppy disk, stored on the expansion RAM, where it was then read by the Famicom. So yes, the original Zelda in Japan had loading screens. In a Youtube video, translator Clyde "Tomamto" Mandolyn, said that this is an aspect of classic Nintendo that Japanese gamers are nostalgic for. They say that the clicking of the FDS when a dungeon in Zelda was loading was like the sounds of stepping down the steps into the dungeon.

In 1986 a new FDS game out of the package cost about $14, and about $3 at a disk kiosk. In comparison a new NES game in 1987 in America could cost up to $60 or even more for new titles, with bargain bin games in the $20 to $30 range! The impermanent state of the media, as well as the reduction in cost of physical hardware, manuals, and logistics meant a reduced cost for consumers. Another aspect of the FDS experience were disk kiosk machines which would allow you to write, and overwrite your games onto the proprietary Nintendo diskettes. Because the disks were two sided you could have a 64KB game on each side of the diskette. I can't help but think that, in a way, the FDS is comparable to the rental industry in the US. Nintendo vehemently battled the legality of the rental system. In Japan the rental industry in terms of media is illegal. The Gaming Historian has the best writeup on that part of Nintendo's history.


The FDS was released in February of 1986 with The Legend of Zelda as one of the launch titles. Less than a year later, in January of 1987, The Adventures of Link was released for the FDS. A Famicom cartridge version of The Legend of Zelda in Japan would not be released until 1994, right before the release of A Link To The Past for the Super Nintendo. In comparison, copies of LoZ did not come into North America until around August 1987 and Zelda II in December of 1988. This delayed release date was beneficial for the US version of the game, there are major regional improvements between the Japanese and North American versions of Zelda II.  The FDS added an extra sound channel, although preference is simply a matter of taste and familiarity. Japanese players, for example, have said that the Zelda Theme music just doesn't sound the same without its iconic bells.



The second reason that the disk system was released may have been due to the hardware architecture of the NES and Famicom. To a certain extent, The NES and Famicom were very similar to the Commodore 64. And for clarification, the Famicom and Nintendo Entertainment System are the same in terms of internal hardware outside of the pin connectors, as far as I know. The only exception I can think of is that European consoles often had slower processor clock speed in order to accommodate the slower refresh rate of PAL television set. The NES CPU is a Ricoh 2A03 MOS 6502 core, a slightly older model than the Commodore 64's CPU the MOS 6510, the NES's CPU is identical to the CPU in the Commodore Vic-20, released before the C64. Here in the States, most of the popular home computers and game machines used a 6502 or some variant of it. Which, incidentally, was designed by a Commodore subsidiary. This is also why, for example, so many bad games were released for the NES. Commodore Games game like Winter Games, Karate Champ, Defender of The Crown, were often quickly ported to the NES by publishers; often terribly. And in some cases they aren't necessarily bad games, just bad ports of Commodore games! The C64 had over 2000 games published for it. If there is a particular NES game that haunted your childhood, check that list to see if it's on there.

While Japan had the Family Computer, we had the Entertainment System. The simplest explanation as to the difference was marketing. Nintendo, entering the home console market in the U.S. after the video game crash had to be very particular in marketing their new system. Indeed, early prototypes and patents of the NES were a personal computer complete with a keyboard and cassette data recorder. As a result of the system being the Family Computer in Japan, it may have been Nintendo's intention all along to release a disk peripheral for their system. And Nintendo did in fact eventually release in Japan a keyboard, BASIC cartridge for the system, as well as the cassette deck for saving levels in "programmable games" such as Excitebike and Wrecking Crew. The save feature was notorious for not doing anything in the U.S. releases. Nevertheless cassettes were slower than diskette and less reliable. So while cassettes were probably fine for programming hobbyists and making your own courses in Excitebike, the average consumer did not have much interest in this component of the Famicom. 

Releasing a watered down computer with the promise of expansions released down the line was a way to release new technology at an affordable cost. By stripping down the Commodore architecture and building a small console Nintendo was able to get video games back into living rooms across America. By offering BASIC for the Famicom, Nintendo was dipping their toes in the hobbiest market without taking much of a risk. Sony did something similar with the Playstation 2 in 2009, offering a Linux development kit for their system.

As mentioned earlier, even though the Disk System sits under the Famicom, there is no underside expansion port like the NES. The Famicom, however, does have a fifteen pin expansion port at the front of the console for third party controllers and accessories, such as the aforementioned ACSII Turbo File. The Famicom Light Gun for example, used this port. It is possible to get the disk system working on a toploading NES, or a toaster NES if you open it up to get access to the pin connectors.

A Youtube user by the name of Chykn80 demonstrated, that you could use the expansion port at the bottom of the NES to run the Famicom BASIC cart with the Famicom keyboard! Some of the pins that the FDS uses in the Famicom were relocated to the expansion port of the NES as well. This Wiki page has mapped the pinouts of the NES expansion port.

Meanwhile, back in Japan, the disk system had become obsolete well before the Famicom's lifespan. Final Fantasy, released in 1987 on cartridge at 145KB, would not have even fit on a Famicom disk. I saw articles online that said by July of 1989 Konami had released a 1-meg cartridge. However this was the MSX release of the game, a Japanese personal computer. The Famicom version still managed to be over a whopping 300kb though. Many publishers also did not release games on the FDS. The low price of the games meant even smaller revenue for the publishers and developers who were already paying Nintendo large licensing fees for the right to publish on the Famicom. The advantage of save state technology was often off set by the fact that games utilizing save states often took much longer to produce and develop than other games, and therefore were more costly. Some third party publishers that did release games on the FDS failed. The most prominent example is Squaresoft who spawned a spiff-off company along with a handful of smaller smaller partners called Disk Original Group (DOG) where they both published and released games for the FDS. In less than two years DOG went bankrupt. Had this venture been successful Final Fantasy would have almost certainly been released for the FDS. Almost certainty developed differently under a different name since Squaresoft might not have been in such dire straights. 

By 1987 battery backed SRAM was finally a reality and we saw games on cartridges with save states for the first time. In fact, battery backed memory might have been engineered by Nintendo in order to solve the save state problem. I was unable to find an earlier example of save states on a cartridge based system or computer BIOS before the Legend of Zelda on the NES. If you have any details or information on battery back memory storage in general please email me! While battery save games were more expensive to produce, this could often be off set by password systems. Metroid, Castlevania, and Kid Icarus were all FDS releases that utilized save states in their Japanese releases. The NES hardware was not originally designed with save states in mind. Battery backed save states in cartridges were almost certainly an afterthought. Games with save states would warn you to hold reset while turning off the system otherwise the NES' CPU would experience power spikes that potentially would write data at random locations, including the battery-powered registers of the cartridge, overwriting and corrupting savegames in the process. Holding RESET would send the CPU into a low-power state which would prevent any power surge from overwriting any register. This issue was resolved in future Nintendo consoles.

I've never been really good at writing endings to long articles or papers, and I really didn't know where or when to end this article. The disk system ended up being a rabbit hole I couldn't find the bottom to. The history of the Disk System and NES are so multifaceted that it is difficult to write about it without writing about technology, politics, or marketing. I hope you find the article interesting or helpful. If you have anything you'd like to add please comment below or send me a line.

Saturday, August 20, 2016

How to Set Up a Civilization V Server

 Last updated: 5/16/22

2K Game's instructions on how to set up a Civ V server are unfortunately lacking, they literally just took a guide that one of the forum members wrote and made it the 'official' guide. While the instructions will get you on right path to initially getting the server SDK running, you will be plagued with crashes and errors. The reality of an SDK existing only for Windows and not Linux should be a forewarning for how buggy this software is going to be. After over a week of trying to figure out how to correctly configure a Civ server, I decided to make my own guide that I can reference and reiterate on in the future. Let me be the first to welcome all of you die hard Civ V fans, and those that are waiting for the two expansions packs to be released for Civ VI so you can actually play the game. Civ VI has now hit the price point of "we've released all the DLC so it's now permanently at least 50% off.

I like to play Civ with my friends, unfortunately we are not all able to play at the same time. Online play without hosting your own server means that other players cannot connect and disconnect at will, and that turn timers are only measured in seconds rather than hours, which can be a problem if you like to take breaks and play a game that can span over a few days. This mode we are going to set up is called Pitboss mode and offers players the most versatility and control when playing online with friends. Below you will find the steps to get started.

Installing for the first time:

1. Ensure that network port 27016 is open to UDP communication on the server's router. Consult your router manual for instructions. The dedicated server does not use Steam's NAT punch-through facilities, so turn on port forwarding and make sure that you are allowing
2. We're going to skip the email settings for now, and come back to that later.
3. Install Sid Meier's Civilization V SDK. This can be found in the Tools section of your Steam Library.
4. Launch Sid Meier's Civilization V SDK.Make sure that *everything* successfully installs. One problem I experienced early on was that the specific version of .NET Framework was not installing successfully; I was getting an error message every time it tried to install. In that case there ended up being an issue with my installation of Windows and I had to reformat.
5. Select Dedicated Server from the SDK Launcher. Once Civ boots successfully you can exit out of the game for now.

Running a Pitboss game without crashing:

- Set the server application to run in fullscreen mode with the flag --window max or --window full with either your Steam Launch Options or in the Window's application's properties screen. For some reason players ending their turn wont be recognized server side until the game window is put into focus on the server. So your screensaver for example will interfere with the game.
- Run at a lower resolution. Also turn down all graphics to their minimum setting.
- Leave the game on the player list; no map view. After all looking at the map would be cheating.
- Use the Server specific launcher in the SDK menu. 
- Run Steam in offline mode.
- On default windows update will restart your computer at 3AM. You will need to configure this so that you don't need to restart your game every other morning or so.

Setting up email notifications: 

The instructions for setting up an email notification on 2K's forums are correct. However you will still have to change some settings in Gmail once you have followed the steps provided. After getting the server options set up in the SDK you will need to log into Gmail and go to Settings>Forwarding and POP/IMAP. Here you need to disable IMAP.

After that you will need to go to the My Account page for your Google account (NOT Gmail, but the Google account itself). From there you need to go to My Account>Sign-in and Security>Connected Apps and Site and then turn on the option that says Allow Less Secure Apps. Otherwise when the turn bot goes to do its thing, you will just get an email instead saying that an unsecure app is trying to access your Gmail. This most likely happens because the emails sent out by your server are unencrypted.

This is what a turn notification looks like:

The game name reflects my frustration of dealing with over a week of server crashes.

Since the SDK tends to crash when Steam is in online mode, Steam notifications aren't really an option. As a result getting email notification on your phone is probably the best way to go about turn notifications as of right now.

The Server Application Get Stuck at "Retrieving Host Information":

On Windows 7 and Windows 10 the only solution to this problem for me was to get the server application to run via the Civilization V SDK launcher by starting it in Steam. Initially when I clicked "Dedicated Server - Launch a Pitboss server" from the SDK launcher nothing would happen. I also noticed that when I right clicked on the icons that I could create a shortcut for all the other applications, but not the CivilizationV_Server icon.

**I have no idea why this works** but you have to cut (not copy) the CivilizationV_Server application from the "..\SteamLibrary\steamapps\common\Sid Meier's Civilization V SDK" directory into the "..\SteamLibrary\steamapps\common\Sid Meier's Civilization V" directory. Then run the SDK launcher again and try to make a server shortcut. This time it should work. Then copy (not cut) the CivilizationV_Server application back into the \Dedicated Server directory. This trick worked for me on two separate machines. In all my years of tech support...

Once you launch the application you should only be on the retrieving host information screen for literally one second. A Windows Firewall popup should appear asking you if you want to allow the application to speak to the Internet.

Setting the Sid Meier's Civilization V SDK application to be run as an administrator might help. However you do not have to do this for the CivilizationV_Server application.

How can I run the game in a cloud server:


Someone by the name of CraftedCard developed a Civilization 5 dedicated server application that is runnable on a headless GPU-less Linux machine. Which means that you can now run the application  on a cloud server with no graphics card (you're still going to have to run the Civ server app using Wine even though the Civ client app runs on Linux). Benjamin Jordanou-Bailey has written an excellent guide on how to do just that here if you need a little hand holding with the setup. This is an excellent guide for someone who is new to spinning up cloud machines and containerization. The readme of CraftedCard's repository also has some interesting note on issues they ran into when developing the application, something you'll appreciate if you got this far. 

At the time of writing this article only Vanilla Civ V works using this method. 


Joining a pitboss game:

If you encounter a bug where a player or two are getting "error joining multiplayer session" it is probably because they don't have some small chunk of DLC that the game was hosted with. In this case it's best to remove that DLC option from the settings menu, even if you aren't using it. Otherwise the game may boot into vanilla Civ. The easiest way to tell if this happens is if you get the loading screen in the background that looks like the Atlas statue, and on game start you wont see tourism or faith.

If you listed your game as private other players will need to type in your IP and hit enter to connect. Otherwise if you are playing on a computer behind the same router or network as the server you will be be able to find the game listed under local network.

Sunday, November 17, 2013

Radio Lab Epidoes on Morality



NPR's Radio Lab show recently did a whole episode on Morality. The highlight of the show was a segment done by Amy O'leary in which she describes a game she played in elementary school called Homestead, described as a cross between Dungeons and Dragons and Monopoly, and how she managed to completely control the game and the other "citizens" (starts at 37:20). You can listen to the whole episode for free here.

Wednesday, July 3, 2013

Exploiting Blogger For Passwords

Disclaimer: I do not recommend doing this, and I am not responsible for any trouble you get in for doing this. This post is for educational purposes only. ETC ETC

While working on my site I stumbled upon a way one could harvest Google account passwords from a simple blog account. By setting up a Blogger page, you have the option to include a Google Friend Connect "Join this site" button widget complete with profile pictures of other members who have joined the site.

Look familiar?

At first, I was hesitant to post this, because this site uses the Google Friend Connect widget. Then I realized that no one has used to feature in almost a year, and that the number of users using the service have dropped off, while traffic to the site remained steady. That doesn't mean one could do this with any other sort of connect feature websites often have (Facebook, Reddit, etc).  Pointing out such flaws will hopefully make such services much more secure in the future, as it will hopefully be corrected in some way.

Clicking on the blue button prompts the user to input a Google, Twitter, or Yahoo account login and password. By simply misdirecting the path of the follow URL, one could then spoof the login page so that the information entered follows through to  malicious site. All one has to do is replicate the Google service login popup. No URL is displayed during the whole process.

Such a method has been used before. Mistyping the Craigslist URL might bring you to a page notifying you that you need to update your version of Adobe Flash. Everything from the color scheme used, the font, and the installer are all reminiscent of what Adobe uses for their flash program.

uh oh...

Of course, the URL supersavings.glidehomes.com should be a giant red flag! Following through with the install will almost certainly lead to viruses/maleware/adware. Another approach of this technique is exploited by the torpig botnet. Which university researches were able to take control of. THey learned that the botnet was spoofing online banking logins for major banks on the infected computers, then sending the login information to whoever owned the botnet. Pretty scary stuff.

Did Bots in Quake 3 "Learn" Pacifism?


Anon supposedly recreated the ending of Wargames


No. A 4chan post has been circulating the Internet recently, claiming that after four years on a server full of bots in a Quake 3 death match game, the bots learned that the best strategy was not to kill each other. Huffington Post jumped on the story, although they later updated the page admitting the story was a hoax. However, they did not explain why the story was fake. A major news outlet jumping on an anonymous 4chan copypasta should be a reflection of the current state of news today. 

The original 4chan thread. Click for large.
A whole thesis was published by ID Software in 2001 detailing the build and intricacies of the bots. Furthermore, in 2005 the Quake 3 source code was released under the GNU license, meaning anyone with a compiler could dig through the code. This should give us the tools necessary to scrutinize these claims.
Let us first take a look first at the original post:

"Quake 3 bots were designed off an artificial neural network.
They would effectively "think" to figure out the best way to do things. 
They would see which tactics worked and which didn't.
The longer they played the more they would learn about you, figuring out your patterns, and this would apply to other bots aswell. For all intents and purposes they were one of the first learning AI in a game.

When I found out that I set up a Quake 3 arena server on my pirating server just to have 16 bots over and over just to see how good they would get. 
I set the server up 4 years ago, it's been running the entire time, I forgot about it until this thread. I'm gonna go check on them."
From page 13 of the thesis on Quake AI:
Although neural networks can be useful in several areas in bot AI they are not used for the Quake III Arena bot.
John Carmack even tweeted a response to the claims being made:


The bots are programmed to react to players given reaction time and accuracy constraints given the particular map. The bots DO have an internal representation of the each map given the "volume" of the map. They are not self learning, and they do not aggregate information for use later. "Learning" might not even be an appropriate context for what the bots do. Instead, having your bots standing around is much more consistent with having a bug in the code. For example, the files could have reached a max size limit, and just stopped.

If each of his bot logs are 256 megs each, and there are 16 bots, loading 8 gigs of ram on a 32-bit computer game is implausible in the first place. Even if this guy had over 8 gigs or ram in his computer, he simply could not load all of that bot data in the first place. While the data could be loaded as needed into the rom, the thesis specifically states that not being able to load such specific data in a fast paced game was a limitation of the game, and why a neural network could not be used. Once again, from page 13 of the thesis:
It is also possible to train a neural network during gameplay, which allows the bot to acquire all kinds of knowledge. However training of a neural network is often a time consuming process. The learning capabilities of the neural network will often have to be limited due to the time constraints in a real-time application like a game.
Finally there are other aspects of the story that are bizarre. Who has a server running for four years and forgets about it? During that time the game never crashed; the electricity never went out? No screenshots or video of this situation exists? The particular conditions should be replicable, and could possibly take much less time than4 years to replicate. I am not saying that such a situation of pacifism is implausible, but this story is clearly a hoax.

Quake has almost always been predominantly a multi-player game, with AI rarely being a focus in lieu of graphics, networking, and game-play elements. The bots could still do some pretty impressive things however. They were able to parse per-determined chat sentences given certain environmental characteristics (p.57).

ex. 1

if
environmental change
then
if
bot wants to
chat
then
choose initial chat
use random strings in chat message
replace synonyms in chat message to add variation
output chat message
endif
endif
 
 ex. 2

if
bot receives a message
then
replace synonyms in the message
interpret message using match templates
if
match is found
then
perform action
else
if
messages is a chat message from another player
then
if
bot wants
to reply to this message
then
find a reply chat
use random strings in chat message
replace synonyms in chat message to add variation
output chat message
endif
endif
endif
endif
As mentioned before, the bots DO react to players given reaction time and accuracy constraints and traverse "nodes" in relation to the current map. In this way, the bots do "think", but they do not "learn". AI programming is still being developed interdependently for contemporary first person shooters, and none of them have replicated anything like this. It is not to say that something like this could never happen. But one should keep in mind the disclaimer pasted across the top of some 4chan boards. "The stories and information posted here are artistic works of fiction and falsehood. Only a fool would take anything posted here as fact."