Infolab wiki
Infolab QuickStart
Contents
Infrastructure
The InfolabComputeServersStats lists our big memory servers. These are first come first serve (nobody is managing the resources). If you are on a tight deadline or you feel that somebody is hogging the machine talk to them or talk to your local sysadmin.
You can access this machines by logging in via ssh with your CSID. Here's an example:
~$ ssh tommy@madmax5 tommy@madmax5's password: Last login: Sun Oct 4 10:54:11 2015 from whale.stanford.edu tommy@madmax5:~$
Using Windows? Use PuTTY or Cygwin.
Which server to pick?
The one that's feeling lonely (shows zero or little utilization). As a general rule of thumb Chris' students should go for raidersX machines and Jure's students should go for madmaxX machines.
How do I set up passwordless/key based ssh?
You do not. It's a limitation of our configuration. You need to supply your password on login. Of course you can be resourceful and do:
~$ ssh tommy@madmax5 tommy@madmax5's password: Last login: Tue Oct 6 00:52:52 2015 from whale.stanford.edu tommy@madmax5:~$ ssh madmax3 Last login: Fri Oct 2 09:05:37 2015 from whale.stanford.edu tommy@madmax3:~$
Did you notice that we never got asked for a password when logging into madmax3? Magic, huh? It could be just Kerberos.
But I hate typing in my password all the time!
No problem. Install Kerberos on your workstation. Here are some installation notes: KerberosMac | KerberosWindows.
Can I login from home?
Depends where home is. Logins over ssh should work from almost 1 any network on Stanford Campus. If you're not on campus you have three options.
1) whale
Login via ssh to whale.stanford.edu first. Whale is on one of our networks so you can ssh to other machines from there. Here's an example session.
~$ ssh tommy@whale.stanford.edu tommy@whale.stanford.edu's password: Last login: Sun Oct 4 10:53:42 2015 from c-76-111-212-54.hsd1.ca.comcast.net tommy@whale:~$ ssh madmax3 Last login: Tue Oct 6 01:17:08 2015 from madmax5.stanford.edu tommy@madmax3:~$ hostname madmax3.stanford.edu tommy@madmax3:~$ exit logout Connection to madmax3 closed. tommy@whale:~$ exit logout Connection to whale.stanford.edu closed.
Hey... and this has an added benefit of only typing your password once. Yay!
2) Stanford VPN
Setup the Stanford VPN connection, establish the connection and you're done.
3) Infolab VPN
Check out the instructions on the VPN page. The benefit of this VPN service is that it looks like regular https traffic... so it should work from most hotels, airports, etc.
Storage options
Storage option |
Mount point |
Good for |
Speed |
Backed up? |
Your home directory |
/afs/cs/u/tommy |
Stuff that matters, e.g. results, code |
Not really |
Daily |
Local hard disk |
/lfs/local/0 |
Temporary files, intermediate results |
Around 150 MB/s |
No |
Network storage |
/dfs/scratchX |
Datasets, things you need accessible across multiple servers |
Up to 450 MB/s, but shared! |
No |
It is not common but a server could have multiple local volumes (think of it as having multiple disks) so check if there is an /lfs/local/1 if you're running out of space.
Long running sessions
Write all the rage that are those screen sessions... mention tmux too.
Compute cluster
Create a python script that outputs something useless. Prep a bash script that runs on the cluster
Hadoop cluster
Q&A
Who is Tommy?
Tommy is his name place holding is his game. His friends are Bill Oddie, Private Tentpeg, and Airman Snuffy. Tommy likes to secretly volunteer for this wiki. The agencies involved can neither confirm nor deny that Corporal Schumuckatelli is involved in this matter.
Since the security was tightened your wireless device might fall onto a network that is considered as the public internet... Which means that you'll appear as if you're not on campus as far as our servers are concerned. (1)