Over 50GB: Difference between revisions

From LWP-Wiki
Jump to navigation Jump to search
(Created page with "So we have contacted you, and you are reading this. Your account is so big, the data doesn't fit in your quota on the new server. How to proceed? 1. Decide what to move else...")
 
No edit summary
 
(5 intermediate revisions by the same user not shown)
Line 1: Line 1:
So we have contacted you, and you are reading this.
So we have contacted you, and we have agreed on what to move where. Here's what will happen now.


Your account is so big, the data doesn't fit in your quota on the new server. How to proceed?
1. Once you've told us what to move and where to move it, our servers will start copying data over.


1. Decide what to move elsewhere.
But even when that is done, you won't be moved unless you log off from all LWPs.
For your (and our) convenience, we have created the file $HOME/migration/du. It lists the sizes of your subdirectories.
We have also created $HOME/migration/proposed. It looks like this:


79828 /research
2. So at some point in time, you log off entirely, from all LWPs, when you leave from work in the evening. You also turn off your computer, lest some process fails to stop and keeps the NFS server thinking you're logged in.
32987 /rawdata
------ cut here - dirs below add up to less than 49000MB -------
9159 /edu
7428 /MyBackups
4113 /.thunderbird
3228 /educonf18


That's just a suggestion. Please feel free to select different directories than we did. But do tell us what to move.
3. The next morning (bar some malfunction) when you log in, your home directory is now on another server. Moreover, the directories marked for <code>/project</code> are no longer where they were. They are now in <code>/project/whatever_name_we_agreed_on</code>.


2. Decide where to move it. Please see [[Project_Directories]].
Please note that
ls /project
will not show anything. Only when you actually access a directory under <code>/project</code> will it be mounted and shown.


3. Once you've told us what to move and where to move it, our servers will start copying data over.
If <code>ls /project/yourdir</code> still doesn't show anything, the most likely cause is that your computer hasn't run for at least 20 minutes since we created the the /project/yourdir share. Your computer must be told about it (once) by the Puppet agent, which runs every 20 minutes.
4. Once the data is copied, we mark your account for migration.
5. At a moment when you are logged off (from all LWPs, not just your own), your account will be migrated.


6. The next time you log in, your home directory will not contain the project directories any more. They will be under /project/whatever_name_we_agreed_on.
===Danger!===


Please not that ls /project will not show anything. Only when you actually access a directory under /project will it be mounted and shown.
<em>Do not access</em> <code>/project/whatever_name_we_agreed_on/moved-dirs</code> until <code>~/moved-dirs</code> have vanished from your home directory.
While <code>~/moved-dirs</code> still exist, the copy under <code>/project</code> is just that: a copy. It may lag behind, or be incomplete, or be currently written by our scripts, or all of these. By accessing it, you now have modifications in both copies. That constitutes a mess, and since we warned you, it is you who gets to sort it out.


When you are migrated, not only will your home directory server be on another server, some data will now be under /project
Once <code>~/moved-dirs</code> have vanished, your account has been migrated, and the new path under /project is the only one. Only then is it fine to use it.
 
===Handy symlinks===
 
Once you have been migrated (and not sooner, see the danger above), you could create symlinks from the old location to the new one:
ln -sT /project/whatever_name_we_agreed_on/somedir ~/somedir
There are programs that will treat a symlink differently from an ordinary file or directory, but most will keep working as if nothing had happened.

Latest revision as of 08:59, 1 July 2018

So we have contacted you, and we have agreed on what to move where. Here's what will happen now.

1. Once you've told us what to move and where to move it, our servers will start copying data over.

But even when that is done, you won't be moved unless you log off from all LWPs.

2. So at some point in time, you log off entirely, from all LWPs, when you leave from work in the evening. You also turn off your computer, lest some process fails to stop and keeps the NFS server thinking you're logged in.

3. The next morning (bar some malfunction) when you log in, your home directory is now on another server. Moreover, the directories marked for /project are no longer where they were. They are now in /project/whatever_name_we_agreed_on.

Please note that

ls /project

will not show anything. Only when you actually access a directory under /project will it be mounted and shown.

If ls /project/yourdir still doesn't show anything, the most likely cause is that your computer hasn't run for at least 20 minutes since we created the the /project/yourdir share. Your computer must be told about it (once) by the Puppet agent, which runs every 20 minutes.

Danger!

Do not access /project/whatever_name_we_agreed_on/moved-dirs until ~/moved-dirs have vanished from your home directory. While ~/moved-dirs still exist, the copy under /project is just that: a copy. It may lag behind, or be incomplete, or be currently written by our scripts, or all of these. By accessing it, you now have modifications in both copies. That constitutes a mess, and since we warned you, it is you who gets to sort it out.

Once ~/moved-dirs have vanished, your account has been migrated, and the new path under /project is the only one. Only then is it fine to use it.

Handy symlinks

Once you have been migrated (and not sooner, see the danger above), you could create symlinks from the old location to the new one:

ln -sT /project/whatever_name_we_agreed_on/somedir ~/somedir

There are programs that will treat a symlink differently from an ordinary file or directory, but most will keep working as if nothing had happened.