The problem I was having was that when I went to talk to the SVN server, it would always ask me for my username and password, there was no option to remember it. I thought this a little strange and upon visiting the preferences dialog (Window->Preferences->Team->SVN) I was greeted by a nice error "Failed to load JavaHL library". Which of course meant I couldn't change any options without it bitching that it wasn't available.
So how do we fix this? First off...
sudo apt-get install libsvn-java
Then, if you're lucky, you should just have to restart Aptana in order for it to now detect it. However, if your machine was like mine and it didn't detect it then you'll need to do the following. Go to where Aptana is installed, for me this was /usr/local/aptana
cd /usr/local/aptana
gedit AptanaStudio.ini
You'll then need to add the following line to the end of this file
-Djava.library.path=/usr/lib/jni
That's technically it, all you should need to do now is save the file, restart Aptana and it'll be working for you. Well, I say you, I mean me :P
So now that JavaHL is installed correctly, when I next talked to the SVN server, it once again, asked me for my login details, however a lovely "Save Credentials" checkbox appeared saving my sanity from inputting my details everytime. :D