SharePoint Online, SharePoint Server, SkyDrive Pro

SkyDrive Pro | PowerShell Script checks and moves folders and files


SkyDrive Pro, PowerShell script, checks and moves folder and files

SkyDrive Pro and PowerShell?

Yes, no discrepancy. I used PowerShell because the script should be customizable. But in turn. Again and again, users have problems with files that do not synchronize with SharePoint (online) or SharePoint on premise.  And the users do not know, how they should solve the problem. sometimes users want to move complex folder structures into a SharePoint library(see remarks on the end of this blog post). And Andre Kieft, Microsoft partner technical consultant from the Netherlands has brought me with his little script on the right track.

If you are looking for a german description, here it is

Appended [09.09.13] Andre Kieft wrote an excellent Blog post on technet

Appended [03.10.13] Due to changes from Microsoft on SharePoint Online, it is now allowed, to sync Files with the extension *.exe und *.dll. I have made these changes also on the Powershell-Script.

Most problems with folders and files that are added in the SkyDrive Pro folder and then cause a synchronization error, are the restrictions on the SharePoint Server 2013:

However, we must distinguish

SharePoint Online (Office 365) blocked file extensions  (19)
f.e:  .ashx, .asmx… .xamlx
SharePoint Server 2013 and SharePoint Foundation blocked file extensions (104)
f.e.. .bas,  .bat,… .wsh
File- und Folder restrictions not allowed characters and extensions (9+20)
f.e.: “&”,”%”, or “#”, …
length of folder or filename folder< 256 , file < 128

Depending of the SharePoint Server a user must look between 48 and 133 restrictions on each folder and filename.

  • Simple copying a file into the SkyDrive Pro folder
  • Or copy entire folder with subfolders and files contained therein.
    SkyDrive Pro synchronizes only.  But the user see sync errors. Microsoft is working on this problem on SharePoint (online), may be you will see in the near future , that it will be allowed to use the “&” character in a filename. But not right now

    Solution

    The PowerShell Script SDPMove.ps1 

    The parameters:

    InputPath the source-folder
    f.e: C:\Test
    OutPutPath the destination folder 
    f.e: “C:\Users\hb\SkyDrive @ myCompany”
    -SPOnline optional parameter
    if it is not specified the script will check against SharePoint Server on premise.
    if specified, the script will check against SharePoint Online
    f.e: –SPOnline
    -Fix optional parameter
    if not specified, the script will only check and make no change
    (but you may see some errors)
    if specified, the script will check, change and

    a) folders will be copied to the destination folder
    b) files will be moved to the destination folder. 

    f.e: -Fix

    -Show optional  parameter

    if not specified, the script will give you minimal messages in the PowerShell console.
    if specified, the script will give you all messages

    f.e:  -Show

    -Language optional  parameter

    if not specified, the script will give messages in german language.
    This time 2 languages implemented: German and English
    Parameter: German, English

    SDPMove, start without parameters

    When it is called, the source directory is processed hierarchically. Here the flow chart

     

     

    image_thumb[2] After the script has been started, the parameters will be checked and then all not allowed extensions will be loaded.
    image_thumb[3] First run: The folder structure will be checked
    image_thumb[4] In the Check Folders & Files part ,all not allowed extensions and characters will be checked and replaced.
    image_thumb[7] the checked folder Structure is written into the destination folder.
    image_thumb[8] Second run: all files are checked.
    image_thumb[9] the individual files are written into the destination folder.

    Special features:

    folder

    Invalid characters are replaced.
    Invalid extensions are replaced with the addition of – HBx. The x stands for a numerical value.
    Folder remain left in the source directory.
    files

    Invalid characters are replaced.
    Invalid file extensions are not replaced, the file will remain in the source directory.
    folder and files Is determined by a change, that a folder or file with the same name exits, the script add –HBx  to the folder or filename. The x stands for a numerical value.
    the script cuts too long folder and filenames.

    You may notice and the Script name SDPMove says: the files will be moved to the destination directory. You may do a copy of the source directory in advance.

     

    Here a small selection, which changes SDPMove does:

    Original Change
    Order&Bill.docx OrderandBilldocx
    Offer&Version&1.docx OfferandVersionand1.docx
    request#1.xlsx request1.xlsx
    report~.docx report-.docx
    Test.dll.txt no change, remains left
    try.asmx no change, remains left

    Download

    Here you can download the zip file with the script

    Unpack you the zip file and place SDPMove. ps1 for example in the folder C:\PowershellScripts

    Start PowerShell and navigate to the path C:\PowershellScripts

     

    Call

    InputPath

    We assume you have a folder containing a complex structure with additional folders and files: "C:\Test"

    OutputPath

    You want to bring the "C:\Test" folder and all subfolders contained in it and all files on SkyDrive Pro: "C:\Users\Benutzer\SkyDrive @ myCompany"

    Then the command is:

    .\SDPMove.ps1 C:\Test “C:\Users\Benutzer\SkyDrive @ myCompany” –SPOnline –Fix –Show –Language English

    Problems when you run? (Execution policy)
    look here and run Set-ExecutionPolicy RemoteSigned   (as adnministrator)

    Script-Changes

    You may do script changes. If you run it against a Sharepoint Server On Premise you have to it. Check the file-extensions with you SharePoint administrator. If he has some more blocked file-extensions, than you have to adjust Init_SPOnPremiseIllegal. The same procedure, if he has allowed some file extensions.

    Remarks

    SDPMove is suitable also for moving complex folder structures. Whether it makes sense to bring a previously existing file folder structure 1:1 for SharePoint, this is question, SharePoint consultants should get here first of all their consulting skills in the game

     

    Feedback and comments welcome

    55 Gedanken zu “SkyDrive Pro | PowerShell Script checks and moves folders and files”

    4. Please disregard the post I made I was able to download the script under the „Download section“ If you see the Solution in this article at the top it makes it seem as the ps1 is there but it appears to be a broken link and it is not really there.

      Thank you all the same I will start testing!!

      Cesar

      Like

      Antworten

    5. Good day, I wanted to say thank you for the post as this is exactly what I am looking for but when I attempt to download the script it is not allowing me to retrieve the file. I have tried this is different browsers as well as another computer. If you can send me a new link to the ps1 that would be great.

      Thank you!

      Cesar

      Like

      Antworten

    6. Hello Hans, Thanks for this blog entry. Is it still valid for OneDrive for Business or are there other tools these days to check local / network files shares before the move to OneDrive for Business?

      Like

      Antworten

    7. Any chance an updated link can be provided? Found your site from a Technet blog and it’s exactly what i’m looking for, but your link is broken to download your script.

      Like

      Antworten

      1. try it again… So this was an old SkyDrive-Link. And Microsoft has changed that. Keep in mind, that you have to adjust the script, because the limitations have been changed!!

        Like

        Antworten

    10. In regards to the percentagecomplete errors, simply replace the $count by -PercentComplete ($Count/$items.count*100)

      So

      Write-Progress -Activity $LMsg.Get_Item(„FolderMove“) -PercentComplete ($Count/$items.count*100)

      This will calculate and display the correct number based on the file or folder count and show that in a correct percentage.

      Like

      Antworten
    11. Pingback: Migrating File Shares to SharePoint Online | dbj().near;

    13. Fantastic. I’ve fought with this script all day, tweaking it to do what I want (e.g. ignore & as SharePoint online now allows this) but KEPT getting fatal errors.
      At last tried it from a different PC and it works perfectly! My Powershell was out of date, or corrupt, or the .NET framework the same. Uninstalled and reinstalled and all is now lovely!
      Thanks – this is going to save me a packet, as we have just been quoted a vast sum for a nice piece of software to do the migration, and now I won’t need it!

      Like

      Antworten
    14. Pingback: Migration | File-Server zu OneDrive for Business oder SharePoint | Hans Brender's Blog

    15. This a great script I would love to be able to run it once and produce a report that would summarizes the number of issues in each category then lists all the files that would change.
      Can anyone help with this?

      Like

      Antworten

    16. In order to avoid false alarms regarding for ‚Found an illegal extension‘, I suggest changing code in lines 190, 195 and in 200 when matching file name with a member of the array with illegal extensions. As an example when file name ’species of wasps.docx‘ is matched to ‚.asp‘ in the present script it will return ‚true‘. That is because in RegEx dot ‚.‘ is one of special characters. In order to avoid this, backslash ‚\‘ character should be added to the beginning of the ‚.asp‘ so that file name is matched with ‚\.asp‘.

      Like

      Antworten

    19. Thanks, I really like this article that provides good information related to move files and folders to SharePoint Online. I tried this LepideMigrator for Documents (http://www.lepide.com/lepidemigratordocuments/) which helps to migrate crucial data from file server and exchange public folders into SharePoint and move the attributes of all file, email, folder etc. and manage the schedule jobs as per your accommodation.

      Like

      Antworten

    20. Just to clarify; the One Drive Business Edition sync client MUST be installed on the computer where this script runs, correct? So the script does NOT talk to the One Drive cloud at all directly but rather it moves local files from one directory to another while checking for size and file names, correct?

      Like

      Antworten
    21. Pingback: OneDrive for Business (formerly SkyDrive Pro) File Size, Number, Type & Character Limits and Restrictions | Keith Tuomi - SharePoint Server MVP

    22. The script creates continual errors and stops due to the percentcomplete and count commands, once the value for count is above 100 it generates errors constantly.
      If you comment out (by adding # at the start of the line) lines 96, 97, 113, 114, 138 & 139 the script will run through to completion. It still reports every single file as ‚Found an illegal extension‘ but this appears just to be incorrectly reported.

      Like

      Antworten

      2. Also if you # out lines 176 & 204, only change, illegal extension and error messages are reported back to PowerShell. Much better if you’re scanning thousands of files 🙂

        Like

        Antworten

    23. I’m having the same issue as Jake running on Windows 7.
      I find if I just continually hit enter the script continues to run until it completes.
      I’ll try commenting out that line, but also getting it on line 97 char:27 (see below)
      I suspect this is the percentcomplete command, because it’s returning a value over 100 it keeps generating an error.

      + Write-Progress <<<< -Activity $LMsg.Get_Item("FolderCheck") -PercentComplete $Count
      + CategoryInfo : InvalidData: (:) [Write-Progress], ParameterBindingValidationException
      + FullyQualifiedErrorId : ParameterArgumentValidationErrorEmptyStringNotAllowed,Microsoft.PowerShell.Commands.Writ
      eProgressCommand

      Like

      Antworten

    24. I am having an issue with the script on 8.1 with the following options „–SPOnline –Fix –Show –Language English“ I am getting a lot of illegal extensions that are clearly ok (.pdf). I think the problem is in the status update code. I am getting:

      Write-Progress : Cannot validate argument on parameter ‚PercentComplete‘. The 346 argument is greater than the maximum
      allowed range of 100. Supply an argument that is less than or equal to 100 and then try the command again.
      At C:\Users\Jake\Downloads\SDPMove\SDPMove.ps1:139 char:98
      + … ercentComplete $Count
      + ~~~~~~
      + CategoryInfo : InvalidData: (:) [Write-Progress], ParameterBindingValidationException
      + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.WriteProgressCommand

      Like

      Antworten
    26. Pingback: Keith Tuomi – SharePoint Server MVP OneDrive for Business (formerly SkyDrive Pro) File Size, Number, Type & Character Limits and Restrictions » Keith Tuomi - SharePoint Server MVP
    27. Pingback: Migration from File Shares to Document Management (SharePoint 2013) using OneDrive (Skydrive) for Business | Cosmin's Hooking testify

        1. Hi. OS: Windows 8.1. SharePoint Online. A filename: „Paolo.doc“. Anyway, when I use I -show on the command line, I get “Found an illegal extension” for EVERY file and/or folder. Thanks.

          Like

        2. I answered to your questions in a previous post, but now my post has disappeared… Have you fixed the issue?

          Like

    31. Pingback: Migrating File Shares to SharePoint Online - Partner Technical Services Blog - Site Home - TechNet Blogs
    32. Pingback: Herramientas para la migración de ficheros a SkyDrive PRO | blogjordisanchez
    33. Pingback: Skydrive Pro-Tipps: Beschränkugen von Sharepoint 2013 berücksichtigen
    34. Pingback: Skydrive Pro als File-Server-Ersatz? Ja, aber – man muss die Limits von Sharepoint 2013 kennen und meistern (Teil 1)
    36. Pingback: Migrating File Shares to SkyDrive Pro - Share-Port - Site Home - TechNet Blogs
    37. Pingback: SharePoint 2013 Nuggets of the weeks 34 & 35 - Ragnar Heil: SharePoint Nuggets - Site Home - MSDN Blogs

    Hinterlasse eine Antwort zu Hans Brender Antwort abbrechen

    Diese Seite verwendet Akismet, um Spam zu reduzieren. Erfahre, wie deine Kommentardaten verarbeitet werden..