Creating a PDF Document Viewer in a HoloLens Application
This article is part of a series with best practices from projects executed by a Microsoft CSE team I’m part of. Have a look at Learnings from developing a HoloLens Application to get the overview of the other posts.
In the HoloLens Application we build in a project, we needed UI to show PDF documents coming from a backend service. Well, actually the customer wanted to show Microsoft Office documents. This is currently not possible in an immersive app. So images or PDF is then the best option, of course without editing functionality. We chose PDF as the standard.
Then the next issue was that there is no PDF renderer available in the standard platform. We chose to use the Paroxe component, available in the Unity Store for a reasonable developer license fee. But you can of course also use another PDF renderer of your choice.
If you just want a solution for this problem, and you’re not interested in how this was build, just download the Document Viewer Unity package from the repo, import it and use it. The rest of this post explains the mechanics of this package.
Loading PDF and render to Texture2D
To keep the implementation of the used renderer separate from the rest of the implementation, we created a PDFDocument class. You can find the complete implementation in the MRTK Utilities repo and you can download a separate Document Viewer Unity package from that repo as well.
In the PDFDocument class we have two separate implementations. One for when Paroxe is included in the project, and one when using images for demo purposes. Using images is default, but just by uncommenting the #define PAROXE_INSTALLED line at the top you can switch on the use of Paroxe.
The main constructor is used to load a file and parse it:
private int _pageCount;
public int PageCount => _pageCount;private bool _isValid;
public bool IsValid => _isValid;
...public PDFDocument(Stream fileStream)
{
if (fileStream == null)
{
throw new ArgumentNullException(nameof(fileStream));
}#if PAROXE_INSTALLED
byte[] content;
using (MemoryStream ms = new MemoryStream())
{
fileStream.CopyTo(ms);
content = ms.ToArray();
_pdfDocument = new Paroxe.PdfRenderer.PDFDocument(content);
_isValid = _pdfDocument.IsValid;
_pageCount = _pdfDocument.GetPageCount();
}
#else
// FOR DEMO PURPOSES ONLY
// We have 2 fake pages to show when Paroxe is not installed
_isValid = true;
_pageCount = 2;
#endif
}
Then a method RenderPageToTexture(page) was implemented to retrieve a Texture2D for a page so it can be displayed in the UI.
// Keep page texture reference so we can clean up on
// Dispose or when new page is requested.
private Texture2D _pageTexture;// Used to enhance the resolution of page textures
private int _ratio = 4;
...public Texture2D RenderPageToTexture(int page)
{
if (page >= _pageCount)
{
throw new ArgumentException($"{page} is out of range.");
} if (_pageTexture != null)
{
// clean up last page texture
UnityEngine.Object.Destroy(_pageTexture);
}#if PAROXE_INSTALLED
// Retrieve the page to render in memory.
using (Paroxe.PdfRenderer.PDFPage pdfPage =
_pdfDocument.GetPage(page))
{
// render the request page texture
_pageTexture = _pdfDocument.Renderer.RenderPageToTexture(
pdfPage,
int)pdfPage.GetPageSize().x * _ratio,
(int)pdfPage.GetPageSize().y * _ratio);
}
#else
// FOR DEMO PURPOSES
// If Paroxe is not installed, we have 2 image pages to render
string imageFileName = $"PdfDemoPage{page}.png";
Stream fileStream = File.OpenRead(
Path.Combine(
Application.streamingAssetsPath,
imageFileName));
byte[] content;
using (MemoryStream ms = new MemoryStream())
{
fileStream.CopyTo(ms);
content = ms.ToArray();
_pageTexture = new Texture2D(1, 1);
_pageTexture.LoadImage(content);
}
#endif
return _pageTexture;
}
Building an MRTK based UI to show the document
There are of course various ways to build a UI to show the PDF document, but we based it on the MRTK SlateBlank prefab as a base. You can be find the complete implementation in the MRTK Utilities repo and and in the separate Document Viewer Unity package from that repo as well. But to explain what we did, let’s go through the basic steps.
Start with the MRTK SlateBlank prefab
We started with the SlateBlank prefab and unpacked it completely to have access to all the parts.
Basic changes to the prefab
Let’s do the basic changes first. Rename the SlateBlank to DocumentViewer. Change the Title in the Titlebar to “PDF Document Viewer”. Now hide (uncheck) the ContentBackPlate and show (check) the ContentQuad.
To make sure that the document viewer pops up at a convenient distance, change the Transform.Position.Z of the DocumentViewer game object to the value of 0.8 and the Transform.Position.Y to the value of 0.25.
Changes to the ContentQuad game object
We have to do some work now on the ContentQuad, the object that will show the PDF content. Make sure the Mesh Renderer, Box Collider, NearInteractionTouchable, HandInteractionPanZoom and Audio Source components are all checked (active).
In Mesh Renderer open the Materials section and as Element 0 replace PanContent (which is the checkered background with “G1” and such on it) with PageMaterial (which is off-white) from Assets/CSE.MRTK.Toolkit/ DocumentViewer/Materials.
Next we want the ContentQuad a bit bigger to properly show a good portion of the PDF page. Set the Transform.Position.Y to -0.26 and the Transform.Scale.Y to 0.64.
In HandInteractionPanZoom of the ContentQuad object also uncheck the Unlimited Pan.
Add page rendering to the DocumentViewer game object
Now select the DocumentViewer game object and add the DocumentController script from CSE.MRTK.Toolkit/DocumentViewer/Scripts.
Drag the ContentQuad game object to the Page Renderer field on the DocumentController script properties pane so it can be referenced by the controller for rendering.
Run the application
To run the application, make sure you have the files Fact-Sheet_HoloLens2.pdf, PdfDemoPage0.png, PdfDemoPage1.png, PdfDemoPage2.png and PdfDemoPage3.png in your Assets/StreamingAssets folder. You can use the ones from the repo.
Now you can run the application to see the first version of the document viewer! You can already zoom in and out on the page. No other interactions are possible. And it’s not using a PDF renderer in the basic setup, as that has to be acquired and added to the project to be used. By default it’s using the image files.
Add buttons for page navigation
Let’s add some buttons that enable paging in the document. We’ll add them to the title bar. Under the DocumentViewer game object you’re editing, open TitleBar | Buttons. You see the two buttons for Follow me and Close. Copy one of them twice and paste it under the Buttons game object, and drag them so they are the first objects under Buttons.
Select the Buttons game object, set Sort Type to Child Order. Set the Transform.Position.X to 0.32. Now click the Update Collection button in the properties. The buttons are now properly aligned in the title bar.
Select Pressable Button (3) and in the Inspector window scroll down to the Button Config Helper (Script). Enter Previous page as the Main Label Text. Click on the dropdown that has the value GameObject.SetActive and select DocumentController and ShowPreviousPage() to change it.
If you want, you can change the used image to something distinctive. See Buttons — MRTK 2 | Microsoft Docs for more information. Left and right arrows are provided in CSE.MRTK.Toolkit/Common/Fonts/ SegoeMDL2IconSet.
Now do something similar for Pressable Button (4) but enter Next Page as Main Label Text and select DocumentController and ShowNextPage() for the OnClick action.
If you run the application now, it will look something like this. Click the buttons to change pages in the document. To learn how to simulate hand interactions, see Input simulation service — MRTK 2 | Microsoft Docs.
More modifications
You can add more modifications, like buttons to go to the last and first pages, a button to show the zoom level and reset it to 100%, a page indicator and more. We won’t go into those details here. Have a look at the Document Viewer Unity package from the repo and learn from the implementation there.
Conclusion
This post explained a bit more how PDF rendering can be implemented in an MRTK-based HoloLens Application. But, the solution now uses demo files from the Assets/StreamingAssets folder. For a real solution you’ll probably want to show documents from a backend. And you’ll need something to select a document. Although we came up with a browser specific for the case of the customer, I’ll leave this part to you 🤓
