Getting data in Power Pages

If you have ever worked with Power Pages, as you will know, there are “two” ways to get data via standard (now we will understand why those inverted commas), and on the other hand two other ways via development.

The standard ways are the basic and advanced forms (basicform, advancedform), in which you can read the data from a record via form, and the other is to visualise the data via entitylist. To be able to customise the standard ones, you simply have to choose the entity from which you are going to display the data and to be able to display them you choose the form or the view you want to show.

On the other hand, for development we can do it in the following ways, either via fetchxml or via the Power Pages API.

The fetchxml is obtained via liquid, and is done in the background, so it’s super positive for not leaving traces in the network in case we need a little more security so that users can’t capture these traces and get in the way. The code would look something like this:

{% fetchxml MyQuery %}
<fetch>
...
</fetch>
{% endfetchxml %}

In case you do it with the Power Pages API It would be something like this:

var responseJson = await fetch([Portal URI]/_api/accounts?$select=name&$filter=accountid eq '{{request.params.id}}')
var response = await responseJson.json()

Cache

Now that we have understood the ways in which data can be obtained, let’s take a look at the Portal’s cache. According to the learn page it states the following: “To improve scalability and performance, Power Pages websites cache data that is queried from Microsoft Dataverse. This caching is done on the application server for all business data and website metadata and is different from browser-based static resource caching or content delivery network caching.” … “This configuration data cache for any table is automatically updated when any record is changed. The automatic cache refresh has a 15 minute service level agreement. Any changes made to a configuration record would automatically be available on the web site within 15 minutes.”

What this means is that the Portal cache lasts 15 minutes, and that means that the data does not come to us in real time, but is “near real time”.

I will now give you some examples to show you when the Portal is refreshed and when it is not, and we will comment on the results:

1. 1. A record is created through an entityform. It automatically appears in the entitylist of records --> The cache is automatically refreshed.
2. 2. A record is created via the API. It does not automatically appear in the entitylist of records. --> The cache is NOT automatically refreshed.
3. 3. A record is created through an entityform. Does not automatically appear in a liquid fetchxml or API Get --> Cache is NOT automatically refreshed.
4. 4. A record is created via the API. It does not appear automatically in a liquid fetchxml or API Get --> The cache is NOT refreshed automatically.

As you can see, the cache works in a particular way and cannot be cleared in an API or manual way. (Well, the only manual way is to be an Administrator and be able to go to /_services/about and clear the cache, or from the layout editor, but I don't think you want to have a person 24/7 clearing the cache).

Let’s explain a little bit about the cache in log collection:

1. 1. The request is made to fetch data.
2. 2. That request is sent via API to where the system hosts our Power Pages.
3. 3. The system first checks its cache to see if it has made that SAME request before (notice that I put SAME in capital letters, it's super important for later).
4. 4. If that request is in the cache, the data is fetched and returned, i.e. our 15min cache acts.
5. 5. If the request is not in the cache, the data is fetched from D365 and the request and the data is stored in the cache.

These 5 steps above work when the data is displayed in any of its entityform, entitylist, api or fetchxml forms.

But then what happened in the step? “A record is created via an entityform. It automatically appears in the entitylist of records —> The cache is automatically refreshed”.

Well, when a record is created via entityform, the cache is cleared so that it can be displayed within an entitylist.

Solution

But let’s get to the point, how can we solve in a fetchxml or in a Get API to be able to show data in Real Time?

If you remember, explaining the cache, in the third step we put: “The system first checks its cache to see if it has made that SAME request before.”

There we have it, we need to create a different request to be able to get the data in Real Time. But… how can we make that request?

Well, the most logical thing is to always set a date that is always dynamic, and thanks to Liquid we are going to be able to get it.

In a fetch:

{% fetchxml MyRequest %}

<fetch>
...
      <condition attribute="createdon" operator="on-or-before" value="{{now | date_to_iso8601 }}" />
</fetch>

{% endfetchxml %}

What we are doing is collecting the fetch with the records whose created on is before right now, that is, this condition is always going to be fulfilled. But if you see, the part of “{{now | date_to_iso8601 }}” will always return a different data, because it returns: “2023-09-17T17:20:20:46Z”. Therefore, the query will always be different and will never be the same, so we are bypassing the cache and we will always get data in “real time”.

The same thing would happen with Javascript, if you put the query of ’’ but with Javascript, you are going to get the data in “real time”.

Summary

I hope this has helped you to be able to get the data in real time, without waiting those 15 minutes of cache, as this can be tedious especially if you need to collect payment data or any data that you need that has been created in D365 and you need it now.

Remember to do a different query each time, I’ve already given you the trick of: ’‘.

But if you have any others you can also share them so I can edit this blog to help more people.

You can always contact me at me@victorsolaya.com, if you have any questions or suggestions.