Powershell SharePoint Get all item data from document libraries – CSOM SharePoint Online

The powershell script below shows how to iterate through all libraries in a web on SharePoint Online and extract filename, version and checked out to name.

#Credentials to connect to office 365 site collection url 
$username = "me@isharepoint.onmicrosoft.com"
$password = "mypass"
$url = "https://isharepoint.sharepoint.com/"
$csvfile = "c:\export.csv"


$securePassword = $password |ConvertTo-SecureString -AsPlainText -force

<#
Write-Host "Load CSOM libraries" -foregroundcolor black -backgroundcolor yellow
Set-Location $PSScriptRoot
Add-Type -Path (Resolve-Path "Microsoft.SharePoint.Client.dll")
Add-Type -Path (Resolve-Path "Microsoft.SharePoint.Client.Runtime.dll")
Write-Host "CSOM libraries loaded successfully" -foregroundcolor black -backgroundcolor Green 
#>

Write-Host "authenticate to SharePoint Online Tenant site $url and get ClientContext object" -foregroundcolor black -backgroundcolor yellow  
$Context = New-Object Microsoft.SharePoint.Client.ClientContext($url) 
$credentials = New-Object Microsoft.SharePoint.Client.SharePointOnlineCredentials($username, $securePassword) 
$Context.Credentials = $credentials 
$context.RequestTimeOut = 5000 * 60 * 10;
$web = $context.Web
$site = $context.Site 
$context.Load($web)
$context.Load($site)
try
{
  $context.ExecuteQuery()
  Write-Host "authenticateed to SharePoint Online Tenant site $url and get ClientContext object succeefully" -foregroundcolor black -backgroundcolor Green
}
catch
{
  Write-Host "Not able to authenticateed to SharePoint Online $_.Exception.Message" -foregroundcolor black -backgroundcolor Red
  return
}

#P.S. You can get Lists with single ExecuteQuery, but keeping the above code to connect to the site so that will be common for all samples
#You can try placing below 37 and 38 lines of below 21st line, it will still get the Lists by making single request to the server
#Get all lists Info
Write-Host "Getting all the list info" -foregroundcolor black -backgroundcolor yellow  
$Lists = $web.Lists
$Context.Load($Lists)
try
{
$Context.ExecuteQuery()
Write-Host "Successfully retrived all the list info" -foregroundcolor black -backgroundcolor green  

}
catch
{
Write-Host "Error while getting the list info" -foregroundcolor black -backgroundcolor Red  
}

#this bit creates the CSV if it does not already exist
$headers = "List", "Name", "Type", "Path", "Version", "Status"
$psObject = New-Object psobject
foreach($header in $headers)
{
 Add-Member -InputObject $psobject -MemberType noteproperty -Name $header -Value ""
}
$psObject | Export-Csv $csvfile -NoTypeInformation



Write-Host "Displaying Lists with in the site started....." -foregroundcolor black -backgroundcolor yellow  
#Get List Title
foreach($list in $Lists)
{
	#if($list.BaseTemplate -eq "101" -and $list.Title -eq "Test1")   
	if($list.BaseTemplate -eq "101")   	
	{
		Write-Host "Started on library: " $list.Title -foregroundcolor black -backgroundcolor yellow

	
		$items = $list.GetItems([Microsoft.SharePoint.Client.CamlQuery]::CreateAllItemsQuery())
		
		$Context.Load($items)
		$Context.ExecuteQuery()
		Write-Host "... Found " $items.Count " items" -foregroundcolor black -backgroundcolor yellow
		
		$totalItemCount = $items.Count
		$count = 0
		foreach($item in $items)
		{
			$status = ""
			$version = ""
			$count = $count+1
			
			Write-Host "... Processing file " $count " of " $totalItemCount
			
			if($item.FileSystemObjectType -eq "File")
			{
				$checkedout = $item["CheckoutUser"]
				
				if($checkedout)
				{
					$checkoutUser = $web.GetUserById($checkedout.LookupId);
					$Context.Load($checkoutUser)
					$Context.ExecuteQuery()
					$status = $checkoutUser.LoginName
				}
				
				$versionData = $item.File.Versions
				$Context.Load($versionData)
				$Context.ExecuteQuery()
				$versionData = $versionData[$versionData.Count-1]
				$version = $versionData.VersionLabel
			}
			
			$hash = @{
				 "List" =  $list.Title
				 "Name" = $item["FileLeafRef"]
				 "Type" = $item.FileSystemObjectType
				 "Path" = $item["FileRef"]
				 "Version" =$version
				 "Status" = $status
				  }
				  
			$newRow = New-Object PsObject -Property $hash
			Export-Csv $csvfile -inputobject $newrow -append -Force
		}
	}
}
Write-Host "Displaying Lists Completed" -foregroundcolor black -backgroundcolor Green




Get a users followed sites using JSOM in SP2013 or SharePoint Online

SP.SOD.executeOrDelayUntilScriptLoaded(loadUserData, ‘SP.UserProfiles.js’);

function loadUserData()

{

var clientContext = SP.ClientContext.get_current();

 

var followingManager;

var followed;

 

 

followingManager = new SP.Social.SocialFollowingManager(clientContext);

 

 

Next we need to get the sites followed by the current user. For this we need to pass the actor ID of 4. For a list of all actor Ids see this.

 

followed = followingManager.getFollowed(4);

clientContext.executeQueryAsync(showFollowed, requestFailed);

​

}
​

function showFollowed() {

for (var i = 0; i < followed.length; i++) {

//get whatever you need from the followed array

}

​}

 

function requestFailed(sender, args) {

$(‘#message’).html(‘Error: ‘ + args.get_message());

}

Configure SharePoint 2013 for App Development by setting up isolated domain

If your not sure what needs to be configured for app development on ​SP2013 on premise or you receive the “Apps are disabled on this site” error, then follow these steps:

  1. Ensure the user profile service application is running and has at least one profile in it
  2. Ensure the application management service app is running.
  3. Ensure timer service is running: net start sptimerv4
  4. Ensure you have created an isoloated domain

SharePoint hosted apps run must run in their own domain. For Visual Studio to be able to deploy apps you need to first setup the domain. To do this run the following powershell scripts in the SharePoint Management Shell. Hopefully they should be self explanatory:

Set-SPAppDomain “apps.isharepoint.com”

Get-SPServiceInstance | where{$_.GetType().Name -eq “AppManagementServiceInstance

Get-SPServiceInstance | where{$_.GetType().Name -eq “AppManagementServiceInstance” -or $_.GetType().Name -eq “SPSubscriptionSettingsServiceInstance”}

 

The below scripts are to setipt the application pools. You need to provide a sharepoint managed account.

$account = Get-SPManagedAccount “domain\user”
$appPoolSubSvc = New-SPServiceApplicationPool -Name SettingsServiceAppPool -Account $account
$appPoolAppSvc = New-SPServiceApplicationPool -Name AppServiceAppPool -Account $account
$appSubSvc = New-SPSubscriptionSettingsServiceApplication –ApplicationPool $appPoolSubSvc –Name SettingsServiceApp –DatabaseName SettingsServiceDB $proxySubSvc = New-SPSubscriptionSettingsServiceApplicationProxy –ServiceApplication $appSubSvc
$appAppSvc = New-SPAppManagementServiceApplication -ApplicationPool $appPoolAppSvc -Name AppServiceApp -DatabaseName AppServiceDB
$proxyAppSvc = New-SPAppManagementServiceApplicationProxy -ServiceApplication $appAppSvc

 

Specify your tenant name by typing the following code in the SharePoint Management Shell:

Set-SPAppSiteSubscriptionName -Name “app” -Confirm:$false

Finally your isolated app domain to your bypass list in Internet Explorer

  1. In Internet Explorer, go to Tools.
  2. Choose Internet options.
  3. On the Connections tab, choose the LAN Settings
  4. Clear the Automatically detect settings check box.
  5. Select the Use a proxy server for your LAN check box.
  6. Choose the Advanced button, and then add *.YourAppDomain.com to the Exceptions
  7. Choose the OK
  8. Choose the OK button to close the Local Area Network (LAN) Settings dialog box.
  9. Choose the OK button to close the Internet Options dialog box.

You may get a 401 when launching you app. For this you’ll need to disable loop back checking:

To set the DisableLoopbackCheck registry key, follow these steps:

  1. Set the

DisableStrictNameChecking

registry entry to 1. For more information about how to do this, click the following article number to view the article in the Microsoft Knowledge Base:

281308

(http://support.microsoft.com/kb/281308/ )

Connecting to SMB share on a Windows 2000-based computer or a Windows Server 2003-based computer may not work with an alias name

  1. Click Start, click Run, type regedit, and then click OK.
  2. In Registry Editor, locate and then click the following registry key:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Lsa

  1. Right-click Lsa, point to New, and then click DWORD Value.
  2. Type DisableLoopbackCheck, and then press ENTER.
  3. Right-click DisableLoopbackCheck, and then click Modify.
  4. In the Value data box, type 1, and then click OK.
  5. Quit Registry Editor, and then restart your computer.

 

Use C# CSOM ClientContext to download file from SharePoint document library or OneDrive for Business

This post will show how to download a file from any document library.

 

Using SharePoint SDK in Windows Phone 8 and the ISF (Isolated File Storage):

The steps:

  1. Get the relative url of the file
  2. Use the ClientContext object to download the file to the ISF
  3. Open the file in the phones office apps

The code:

Assuming you have the Item object:

string fileRef =

Stream inputStream;

private void BMSStackPanel_Tap(object sender, System.Windows.Input.GestureEventArgs e)

{

loadingProgressBar.IsVisible = true;

//get the relative url of the file

string fileRef = item[“FileRef”].ToString();

//get just the filename part

string fileName = fileRef.Substring(fileRef.LastIndexOf(“/”)+1);

 

Microsoft.SharePoint.Client.File.OpenBinaryDirect(Context, fileRef,

(object eventSender, OpenBinarySucceededEventArgs eventArgs) =>

{

if (eventArgs.Stream != null)

inputStream = eventArgs.Stream; //instantiating stream using the sharepoint file stream

Deployment.Current.Dispatcher.BeginInvoke(() =>

{

using (IsolatedStorageFile ISF = IsolatedStorageFile.GetUserStoreForApplication())

{

//create a local file with the same name using the input stream and saving through the output stream

using (IsolatedStorageFileStream outputStream = ISF.CreateFile(fileName))

{

try

{

byte[] bytes = new byte[inputStream.Length];

inputStream.Read(bytes, 0, (int)inputStream.Length);

outputStream.Write(bytes, 0, bytes.Length);

outputStream.Flush();

 

}

catch (Exception ex2)

{

}

}

loadingProgressBar.IsVisible = false;

LaunchFile(fileName);

 

 

}

});

}, null); //im passing null for the failed event. this should be handled properly though

}

private async void LaunchFile(string fileName)

{

var file = await ApplicationData.Current.LocalFolder.GetFileAsync(fileName);

await Windows.System.Launcher.LaunchFileAsync(file);

}

 

 

Using Client OM in C#:

Assuming you have the Item object:

//get the URL of the file you want:

var fileRef = item[“FileRef”];

//get the file contents:

FileInformation fileInfo = File.OpenBinaryDirect(context, fileRef.ToString());

using (var memory = new MemoryStream()) {

byte[] buffer = new byte[1024 * 64];

int nread = 0;

while ((nread = fileInfo.Stream.Read(buffer, 0, buffer.Length)) > 0) {

memory.Write(buffer, 0, nread);

}

memory.Seek(0, SeekOrigin.Begin);

// … here you have the contents of your file in memory,

// do whatever you want

}

 

In JavaScript:

  1. Globally declare the content and web objects as shown below.

    var context = SP.ClientContext.get_current(); //gets the current context

  2. var web = context.get_web(); //gets the web object

Now write the function to read a file in a Document Library.
function readFile() {

context.load(web);

context.executeQueryAsync(OnreadServerRelativeURL, readFileFailure);

}

function OnreadServerRelativeURL() {

var fileUrl = web.get_serverRelativeUrl() + “/MyLibrary/MyTextFile.txt”;

$.ajax({

url: fileUrl,

type: “GET”

})

.done(Function.createDelegate(this, readFileSuccess))

.error(Function.createDelegate(this, readFileFailure));

}

function readFileSuccess(data) {

//do whatever you need to with the data

alert(data);

}

function readFileFailure(sender, args) {

debugger;

alert(‘Failed to read a file. Error:’ + args.get_message());

}

 

Use CSOM ClientContext to fetch documents from users personal OneDrive for Business

Say I want to provide the user a list of their personal files in their OneDrive for Business site – this is how to do it.  Note: this is for OneDrive for Business, not the standard OneDrive – for that see this article.

In essence each users personal files are in their own site collection. So we need to get the web url. You can see what the web url should be by browsing to your own OneDrive in the browser.  In my example Im going to use:

https://isharepoint-my.sharepoint.com/personal/luqman_isharepoint_onmicrosoft_com

So that makes sense but what we actually need is the users own personal url, not mine.  For this we need to get the current user object then fetch their email address.  Replace any periods and @ symbols characters with underscores.  For this we can use the main tenant url as we dont current know what the personal site collection url is:

ClientContext  Context = new ClientContext(https://isharepoint.sharepoint.com);​

Authenticator at = new Authenticator();

at.CookieCachingEnabled = true;

Context.Credentials = at;

web = Context.Web;

Context.Load(web);

currentUser = web.CurrentUser;

Context.Load(currentUser);

Context.ExecuteQuery();

string email = currentUserEmail;

 

The code above has been changed a bit from the actual to make it a bit clearer but you should get the idea of whats going on. Also its for windows phone 8 app, for a sharepoint you wont need to worry about the authentication.

OK, now that we have the email address we need to replace the characters to make it the same as users personal folder url:

email = email.Replace(“.”, “_”).Replace(“@”, “_”);

 

Next we get a ClientContext to the users personal site collection of:

https://isharepoint-my.sharepoint.com/personal/”+email+”/”);

Once we have that we need to access the Documents library. We do this jus the same way we would with any other document library and there are hundreds of articles online showing how to query list and libraries.

Develop Windows Phone 8 (WP8) SDK to fetch data from SharePoint Online or 2013

So you want to have a go at developing native windows phone apps that interact with SharePoint Online or even SharePoint 2013 on-premise (as long as your device browser has access to the site)….lets begin!

Pre-reqs:

  • OS – Windows 8/8.1 Pro or better
  • Windows Phone 8 or 8.1 SDK
    • If you go with the Windows Phone 8 SDK please ensure you update it to the current Update 4.  The older versions have issues debugging.
  • SharePoint SDK for Windows Phone

 

Create the project

Open Visual Studio Express for Windows Phone.

Click File > New Project > Visual C Sharp > Windows Phone > Empty SharePoint Project

 

I find it easier going with an empty SharePoint project and adding what you need rather than the SharePoint list project and then having to remove unnecessary stuff.

 

Explore the created files and folders

In particular App.xaml, the assets icons and the AppManifest in the properties.

 

Make the UI

OK so we get a rather plain looking xaml screen (MainPage.xaml) to start with. Lets make it look better by adding a pivot control or panorama. If you don’t know what they are check out this msdn blog.  My personal preference is a pivot control as i think the screen look a lot cleaner with it.  Im going to add 4 pivot items to the pivot control: Home, News, Documents, Sales thats because i have this data in sharepoint and want to show it in the app on a user interface of my choice rather than the standard boring sharepoint app which just shows standard lists.

<phone:Pivot Title=”My app” x:Name=”MainPivot” SelectionChanged=”MainPivot_SelectionChanged” >

<phone:PivotItem Header=”Home”>

<phone:PivotItem Header=”News”>

<phone:PivotItem Header=”Docs”>

<phone:PivotItem Header=”Sales”>

</phone:Pivot>

Im not going to go into the details of creating the UI but in essence the home pivot will have a nice looking grid with buttons and icons to switch the pivots. In the code above ive highlighted the SelectionChanged event. Since we have a lot of data in sharepoint we dont want to load all the tabs at once instead they will be loaded on demand.  Ive also added a progress indication to the xaml which will be set with IsVisible=true in the codebehind when loading content:

<shell:SystemTray.ProgressIndicator>

<shell:ProgressIndicator IsIndeterminate=”True” IsVisible=”True” Text=”Loading data …”

x:Name=”loadingPr

ogressBar” />

</shell:SystemTray.ProgressIndicator>

 

Id strongly recommend switch between Blend and VS when creating the UI. Blend is a lot easier.

 

 

Authenticate with SharePo

int

I want to login the authentication piece out of the way for the user right at the start.

 

For this ive added a Context property to the code.

 

private ClientContext m_Context;

public ClientContext Context

{

get

{

if (m_Context != null)

return m_Context;

m_Context = new ClientContext(“https://isharepoint.sharepoint.com”);

Authenticator at = new Authenticator();

at.CookieCachingEnabled = true; //Allows authenticator to save cookies for future usage

m_Context.Credentials = at;

return m_Context;

}

}

Ive added a Loaded event:

private void TitlePanel_Loaded_1(object sender, RoutedEventArgs e)

{

web = Context.Web;

Context.Load(web);

//you cannot authentication on the main thread, so call a new thread

//so you push the clientcontext loading into a new thread

System.Threading.ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(LoginThreadCallback), Context);

}

private void LoginThreadCallback(object s)

{

var context = (ClientContext)s;

context.ExecuteQuery();//start thread work to authenticate and load sp lists

_syncContext.Post(

new SendOrPostCallback(

delegate(object state)

{

LoginThreadComplete();

}), null);

}

private void LoginThreadComplete()

{

loadingProgressBar.IsVisible = false;

}

 

Binding Data

Now that we have the Context object authenticated when the MainPivot_SelectionChanged event is triggered we can use that Context to load the data and bind it onto out PivotItems.

For example on my News pivotitem ive added a longlistselector with an itemtemplate of how i want each article to appear:

<phone:LongListSelector x:Name=”NewsList” ItemsSource=”{Binding ListData}”  >

<phone:LongListSelector.ItemTemplate>

<DataTemplate>

<StackPanel Margin=”0,5″>

<StackPanel Orientation=”Horizontal” Background=”#80FFFFFF”>

<Image Margin=”5,5″ Width=”120″ Height=”80″ Source=”{Binding ImageSrc}” />

<StackPanel>

<TextBlock Text=”{Binding Title}” Width=”300″ TextWrapping=”Wrap” FontSize=”{StaticResource PhoneFontSizeLarge}” />

<TextBlock Text=”{Binding ArticleDate}” />

</StackPanel>

</StackPanel>

</StackPanel>

</DataTemplate>

</phone:LongListSelector.ItemTemplate>

</phone:LongListSelector>

 

Next create a class (Model) to represent a News article:

public class Article

{

public string Title { get; set; }

public BitmapImage ImageSrc { get; set; }

public string ArticleDate { get; set; }

}

Thats our model complete.  In the below code we fetch the data and bind it to the longlistselector with the x:Name of NewsList.  Each article will also have an image from the PublishingImages folder which will have to be downloaded the Isolated File Storage and then added to the xaml.  Ive skipped out that part but theres lots of online resources for it.

private void LoadNewsArticles()

{

loadingProgressBar.IsVisible = true;

web = Context.Web;

spNewsList = web.Lists.GetByTitle(newsList);

Context.Load(spNewsList);

CamlQuery query = new CamlQuery();

query.ViewXml = “<View><Query><Where><IsNotNull><FieldRef Name=’HomePageText’ /></IsNotNull></Where><OrderBy><FieldRef Name=’Modified’ Ascending=’FALSE’ /></OrderBy></Query><ViewFields><FieldRef Name=’HomePageText’ /><FieldRef Name=’PublishingRollupImage’ /><FieldRef Name=’Modified’ /></ViewFields><RowLimit>10</RowLimit></View>”;

spNewsItems = spNewsList.GetItems(query);

 

Context.Load(spNewsItems);

 

System.Threading.ThreadPool.QueueUserWorkItem(new System.Threading.WaitCallback(NewsThreadCallback), Context);

}

 

private void NewsThreadCallback(object s)

{

var context = (ClientContext)s;

context.ExecuteQuery();

//send the data to the main thread so that it can update the UI

_syncContext.Post(

new SendOrPostCallback(

delegate(object state)

{

SetNewsData();

}), null);

}

 

protected void SetNewsData()

{

if (spNewsItems.Count > 0)

{

//create list of news articles

List<Article> source = new List<Article>();

foreach (ListItem item in spNewsItems)

{

if (item[“PublishingRollupImage”] != null)

{

string imagesrc = item[“PublishingRollupImage”].ToString();

imagesrc = imagesrc.Substring(imagesrc.IndexOf(“/”));

imagesrc = imagesrc.Substring(0, imagesrc.IndexOf(“\””));

string filename = imagesrc.Substring(imagesrc.LastIndexOf(“/”)+1);

 

DownloadFile(Context, imagesrc, filename);//download image to isolated from storage

BitmapImage image = ThumbImageFromIFS(filename);//getting image file from Isolated File Store

 

string homepagetext = item[“HomePageText”].ToString();

if (homepagetext.Length > 35)

{

homepagetext = homepagetext.Substring(0, 34) + “…”;

}

//add news article to the news articles list

source.Add(new Article() { Title = homepagetext, ImageSrc = image, ArticleDate = DateTime.Parse(item[“Modified”].ToString()).ToShortDateString() });

 

}

}

//set the long list selector source

NewsList.ItemsSource = source;

loadingProgressBar.IsVisible = false;

}

}

 

Thats all folks! Hopefully that all makes sense, if not leave a comment below and ill try to respond.

Link to create new documents in Office Online and OneDrive – WOPIFRAME

You can create hyperlinks to the create new document functionality in office online using the following URL structure and template type parameter:

Word:
https://atentant-my.sharepoint.com/personal/[user_folder]/_layouts/15/CreateNewDocument.aspx?SaveLocation=https://atentant-my.sharepoint.com/personal/[user_folder]/Documents&DefaultItemOpen=1&Source=https://atentant-my.sharepoint.com/personal/[user_folder]/Documents&TemplateType=1

 

Excel: Same with TemplateType=2

Powerpoint: Same with TemplateType=3

Onenote: Same with TemplateType=4

 

The folder/url doesnt have to be pointed at the users onedrive. it could be any folder on any site.

 

Took a while finding this. Leave  a comment if you found this useful!

SharePoint App Permissions

Apps for SharePoint have permissions just as users and groups do. This enables an app to have a set of permissions that are different from the permissions of the user who is executing the app. You must request, in the app manifest file, the permissions that an app needs to run. The user who adds the app must grant these requests, and the user can only grant permissions that he or she has as a user. The grant must be for all the requested permissions or none of them to simplify the management of permissions for users and developers. (The app principal always has full control rights to the app web, so it only needs to request permissions to SharePoint resources in the host web or other locations outside the app web.)

What are Host Webs and App Web

SharePoint 2013 introduces the concept of ‘host webs’ and ‘app webs’.

Apps are deployed to the App Web.

Apps are installed to the Host Web.

 

What are App Webs

In SharePoint 2010 when creating a webpart you could create and use and SharePoint components such as lists, libraries or workflows just about anywhere in the scope of where the webpart was being activated.

SharePoint 2013 adds an additional level of isolation (for security) whereby if an App is to need a SharePoint component such as list then it must contain this list within the apps own subweb – that app’s own App Web.  App Webs are usually inside the same site collection as the Host Web, except in tenant environments where they are in the App Catalog.  Apps that require their own web are reffered to as SharePoint hosted apps.

Apps that dont need access to any SharePoint objects such as lists, can exists without an App Web. e.g. those webparts which consume data from webservices etc. These are referred to as Provider Hosted Apps.

See example below where App 1 doesnt use any SharePoint component and therefore has no App Web, whereas App 2 uses lists and workflow and requires its own App Web:

post30

Also, although the app web is typically in the same site collection as the host web it will have its own isolated domain.

For example, suppose that an app, with SharePoint components beyond
just the UI elements that can be deployed to a host web, is installed on
a host website at the following URL:

https://www.contoso.com/sites/Marketing

The app for SharePoint will be deployed to a newly created website with a URL like the following:

http://app-bdf2016ea7dacb.contosoapps.com/sites/Marketing/Scheduler

Note that this URL has the following structure:

https://App_Prefix-App_ID.App_Base_Domain/Domain_Relative_URL_of_Host_Web/App_Name

  • Domain_Relative_URL_of_Host_Web is the relative URL of the parent host web, in this case sites/Marketing.
  • App_Name is the value of the Name attribute of the app element in the appmanifest.xml file.

 

Why to apps have their own web

Microsoft states two reason for requiring App Webs for Apps. Both are for security Enforcement of App permissions (see blog post) and Protecting against Cross-domain scripting attacks.

For more detailed information see the MSDN whitepaper