More Details - HANA Extension Nodes for BW-on-HANA

[klaus_nagel]

This blog provides additional details about the new concept in HANA to manage “warm“ data in BW. It is follows up on my blog where I initially introduced this idea: Update – Data LifeCycle Management for BW-on-HANA

What are deployment options for HANA Extension Nodes?

There are basically three different deployment options for extension nodes in HANA system for BW. Which option you choose depends on your landscape, the sizing for the amount of “warm” data in your system, BW release, HW partner, … and, of course, the timeline.

Why does it work for BW?

The standard HANA sizing guidelines allow for a data footprint of 50% of the available RAM. This ensures that all data can be kept in RAM at all times and there is sufficient space for intermediate result sets. These sizing guidelines can be significantly relaxed on the Extension Group, since “warm” data is accessed

• less frequently,
• with reduced performance SLAs,
• with less CPU-intensive processes,
• only partially at the same time.

The BW application controls and understands the access patterns to BW tables and derives appropriate partitioning and table distribution for “warm” tables. This way BW ensures that a “warm data” table is not loaded completely to memory, but only partially due to efficient partition pruning. The load to memory of the much smaller table partitions is not critical in the usual BW operations (batch load processes).

Based on the modelling object type BW can automatically provide a very good default for the “warm” setting.

• Up to 50% of BWs data can be classified as “warm” (experience with “non-active” data concept)
• access to “warm” tables is partition-based in >95% of all cases (write(=merge)&read)
• data in “warm” tables is part of batch processes in most cases (load-to-memory not critical)
• query access to “warm” data will be significantly slower – must be accepted/part of the deal

How to classify BW objects as “warm”?

The classification of a BW object as “warm” is part of the modeling task in the corresponding modeling UI. The default for all objects is “hot”.

• A newly created object classified as “warm” has all its database tables created on the “extension” node(s)
• An object containing data does not change the location of its tables immediately during object activation, but only changes the metadata of the object. To move the tables there are two alternatives:
  • Execute a table redistribution using the SAP DWF DataDistributionOptimizer (DDO) – this can be seen like a regular house-keeping action,
  • Use transaction RSHDBMON to move single tables/partitions manually.

What type of objects can be classified as “warm” in BW?

This paragraph describes which BW objects can be classified as “warm” and in which BW release the option is available. It does not mean that all these objects necessarily should be classified as “warm” – it depends on the individual use case.

BW Object	Available release	Comment	Caution
InfoCubes	not available	Please look at the options for advanced DSOs
Classic DSOs (exception see below)	not available	Please look at the options for advanced DSOs
DataSources/PSA tables	BW7.4 SP10	A PSA table can be classified as “warm”. PSA tables are partitioned grouping together one or more load requests. Load operations only change the latest partition --> small amount of data for the MERGE process. Extract operations only use the latest partition in most cases (delta loads).
Write-optimized DSOs	BW7.4 SP10	See PSA comment	Only write-optimized DSOs with usage type Corporate Memory should be classified as “warm”. I.e. no reporting access, no heavy look-up usage
Advanced DSOs w/o Activation	BW7.4 SP10 & BW/4HANA	Partitioning and access similar to PSA	See w-o DSO
Advanced DSOs w/ Activation	BW7.5 SP01 & BW/4HANA	Load and Extract patterns are request/partition-based – similar to PSA tables	DSO-activation needs to load and process the complete table in memory è only aDSOs should be classified as “warm” with very infrequent load activity; use RANGE partitioning of the aDSO where possible to allow pruning
Advanced DSOs with reporting access	BW7.5 SP01 & BW/4HANA	Load patterns are request/partition-based – similar to PSA tables	Query read access may load the complete table (all requested attributes/fields) to memory and query processing may be very CPU-intensive. Only classify objects with Very infrequent reporting access Highly selective access (few fields, selective filters hitting RANGE partition criterions if available) Relaxed performance expectations due to load to memory & less CPU
RANGE- Partitions of Advanced DSOs	BW/4HANA	Selected RANGE partitions of aDSOs can be classified as “warm”. Load and Read patterns are request/partition-based – similar to PSA tables.	DSO-activation does partitioning pruning and loads and processes the complete partitions to memory --> only aDSOs partitions should be classified as “warm” with very infrequent load activity

What is the impact for the HANA system?

A HANA system with extension node(s) first of all looks and behaves like a standard HANA scale-out system. All operations and features&functions work as before (like system replication, …).

However there are a few things that should be considered:

• The HANA system can now store more data, which has an impact on backup and recovery times. Especially the higher data volumes on the extension node(s) may now dominate the backup and recovery times – these depends on the hardware for the HANA system.
• Forced unloads are now very common on the extension node(s). On the “hot” nodes many unloads are a sign of insufficient sizing.
• In option 3 and – possibly depending on the choice of hardware – also in option 2, the setup of High Availability using host-auto-failover may need to be adjusted. If no dedicated standby for the extension node exists, it may be necessary to explicitly fall back to the original configuration as soon as a failing node is brought online again.
• For non-BW data the classification “warm” with the re-location to the Extension Node(s) is not supported. If non-BW data is stored in the same HANA DB this data has to be located on the classic nodes.

When will the new concept be available?

Options 1 and 2 are generally released since the Datacenter Service Point (DSP) of HANA SP12.

Offerings for option 3 are still under discussions.

[former_member192508]

Thanks Klaus,

Please could you confirm I understand your first diagram correctly?

- Simple      : additional warm data capacity = 1 x extension node RAM

- Advanced : additional warm data capacity =  2 x extension node RAM

- Special     : to be advised by HW vendor ....

Regards,

Robin.

[BenedictV]

Hello Klaus,

Nice document. The options for setting up data tiering are getting wider and simpler.

Here you mentioned that ....

• "A newly created object classified as “warm” has all its database tables created on the “extension” node(s)"

this classification that is mentioned , is it the 'extended table' option already available or is it something new for the extension node option?

[klaus_nagel]

Hi Benedict,

the modeling classification will be like the "extended table" flag you already see in the system. But the option will be called different, since "extended" refers to the ExtendedStorage used for HANA Dynamic Tiering.

Regards, Klaus

[klaus_nagel]

Hi Robin,

if you add the Extension Node your calculation is correct. If you re-label an existing node as Extension Node you have to substract "0.5 x RAM" from the additional capacity of course.

Regards, Klaus

[former_member192508]

Thanks Klaus, that has clarified the calculation. You have assumed 1TB nodes, so in your simple example you get 4 x 0.5 TB data capacity = 2TB. Convert one to a 1TB capacity warm extended node, you get 1.5TB HOT and 1 TB WARM total 2.5TB capacity.

[kevin_small]

Hi Klaus,

Thanks for post, very interesting. I have a question about the positioning of these extension nodes.

Say I have bought a 7TB BWoH license.  Are extension nodes positioned as:

1. A cheaper way to purchase hardware, because my 7TB license is distributed across (say) 5TB of hot hardware and 2TB of warm (cheaper) hardware.  But my total memory is still 7TB.
2. A way to add more data, because my 7TB license is for hot, and I can add (say) an additional 2TB of warm data on the cheaper hardware, without "using up" my 7TB hot license.  So now I get 9TB of memory.

I appreciate you may not be able to speak about licensing specifics, but if there is a general positioning of this technology I'd be interested to know.

Thanks,

Kevin.

[glen_spalding]

hi klaus

in the concept above, how would the tables be assigned to the extension nodes ?

is this something SAP are endorsing ?

thanks

[klaus_nagel]

Hi Glen,

you model the data "temperature" in the BW modeling UIs. Based on this the generated tables are automatically created on the right nodes.

Regards, Klaus

[glen_spalding]

thanks klaus,

can i qualify then, that this has nothing to do with the current table re organization and distribution process, currently in HANA, using the DWF - table groups, hosts, partition sizes etc ...

and, as you mentioned, this functionality is not currently present. so are you saying, we may see a checkbox or something to put the table of that "extension" node.

thanks

[klaus_nagel]

Hi Glen,

please see the chapter "How to classify BW object ..." above. In the background table groups are used to pin tables to certain nodes (e.g. extension nodes). But the BW modeler does not need to know this.

The modeling UIs already today allow you to mark objects for "Dynamic Tiering/Extended Storage" - this will be changed to allow for the "warm" data modeling for the Extension nodes.

Regards, Klaus 

[richard_bremer]

Hi Kevin,

basically, both answers are correct: if you take a given HANA scale-out setup for BW, the extension node concept allows you to load more compressed data into that setup than traditional BW sizing allows. You view this as a lowering of the cost per TB of data, both in terms of license and in terms of hardware.

Best regards,

Richard

[Former Member]

Hi Klaus,

Thanks for the post. I have a couple of questions.

1. The ratio of Hot to Warm for simple and advanced options seems to be significantly less than the allowed ratio with Dynamic Tiering. i.e., Dynamic Tiering can be sized 4 or 8 times HANA memory depending on the node. Is there a hard limit for the 'special' deployment option?

2. The Data Distribution graphic indicates that moving 30-50% to warm data is not realistic, however, later in the blog the statement is made that 50% is achievable. I guess it depends on each customers, but it would be good to clarify this statement.

3. This architecture is designed purely for a BW on HANA environment. What is the direction for customers with mixed scenario's with BWoH and Native HANA. Can an architecture of extension groups (for BWoH) and dynamic tiering (for Native with DLM) co-exist. Is there an intention to expand the concept of extension groups to Native HANA?

Regards, Gary

[klaus_nagel]

Hi Gary,

ad 1) Dynamic Tiering is based on a disc-based, columnar storage, the Extension Nodes are full in-memory HANA nodes. It means all data that is processed at a given time needs to fit into memory. Looking at typical BW deployments and BWs partitioning schema an "overload" of a factor of 4 is reasonable, anything beyond this is highly customer specific.

ad 2) the hot-warm data distribution is very, very customer specific. I have seen systems with less than 10% as "warm", but also systems with close to 70%. Typical are probably anything between 30 and 50%.

ad 3) the fact that this concept works heavily relies on BWs partitioning and access patterns. To describe these rules in a way that other customer-owned applications/scenarios adhere to this, is fairly complex. That's why we start with the controlled access of BW now and see how this evolves. The Extension Node concept and the Dynamic Tiering concept can co-exit in a HANA instance (however in BW you have to decide which one you use).

Best regards, Klaus

[Former Member]

Hi Klaus,

Thanks for the response. That clarifies my understanding.

Kind Regards,

Gary

[Former Member]

Hi Klaus

One question: Is this approach only available for scale-out scenarios?

Thanks and regards

Florian

[klaus_nagel]

Hi Florian,

this approach works also for a 1+1 system: 1 standard node + 1 extension node.

Best regards, Klaus

[PhilMad]

Hi Florian,

Also check Klaus' reply to my post in the first part of this blog.

Cheers, Philipp

[glen_spalding]

hi klaus

i know you mention this works well for BW, due to the partitioning, and there is much information here inter-twined with partitioning, which i truly understand, however, it looks to me that this technology is not dependent upon partitioning. this is this correct ?

thanks

[klaus_nagel]

Hi Glen,

the "trick" behind this is, that you never process all your data allocated to the Extension Node at the same time, but always only a comparably small part so the data can be moved in and out of RAM. If your tables are partitioned and your access provides the right filters to enable partition pruning, this works very well.

Since BW partitions the generated tables in most cases automatically and according to its access patterns it is pretty save to have more data located on the Extension Node than RAM is available.

So partitioning helps a lot, but indeed is not necessarily required.

Best regards, Klaus

[glen_spalding]

absolutely, thanks

glen

[torsten_wirth]

Hi Klaus,

I'm not sure if I'm getting this right:

Above there are several objects mentioned which are already included in 7.4 SP10 but there is no additional modeling option, is it?

That means only the normal settings 7 = early unload / 5 = no early unload and the manual unload options are available?

Regards,

Torsten

[nicholas_chang]

Hi klaus.nagel

for table classification, is RSDU_TABLE_CONSISTENCY + optimize redistribution plan as per note 214373 (table distribution for BW ) works here,  apart from DDO and transaction RSHDBMON?

As both are following the rules on SYS.TABLE_PLACEMENT table.

Thanks,

Nicholas Chang

[klaus_nagel]

Hi Nicholas,

the configuration of the table-placement rules is described in this note: 2343647 - Howto configure HANA for the BW extension node.

Please keep in mind that HANA SP12 is required and for BW note 2317200 should be applied.

Best regards, Klaus

[nicholas_chang]

Hi Klaus,

Thanks for the reply. Yes, i'm awared of the note 2343647. My question is on existing BWoH scale-out, we'll run RSDU_TABLE_CONSISTECY and "optimize table redistribution" for table classification in HANA studio. If we added an extension node, do we need to include DDO or it is sufficient to just run RSDU_TABLE_CONSISTECY and optimize table redistribution in hana studio?

Thanks,

Nicholas Chang

[Former Member]

hi nicholas,

can you reword

[konstantinos_stergios]

Hi Nicholas,

The redistribution itself can be either done with the native Landscape Redistribution from HANA Studio (that I assume you are referring to) or with DDO. Please check also SAP Note 2317200 (Data lifecycle management for BW on SAP HANA and extension nodes) with regard to the options to move data.

Both, native Landscape Redistribution and DDO, will work with the currently available BW objects. With regard to the new object, the range-partitioned advanced DSO with partition-wise classification as hot or warm data, DDO is planning to include its handling with its next release, whereas for the native Landscape Redistribution it has not been defined when it will be included. Also, DDO will provide an additional algorithm to move only wrongly placed hot/warm objects instead of rebalancing the complete landscape, which will save runtime once you only have to fine tune your landscape.

DDO is part of the SAP HANA Data Warehousing Foundation (DWF). You can find information related to it in SAP Note 2092669 (Release Note SAP HANA Data Warehousing Foundation). Please check the DDO related attachment, as well as the SAP Notes mentioned within this Note. Documentation can be found under the following link: http://help.sap.com/hana_options_dwf

Please note that checking and correcting the table classification entries with report RSDU_TABLE_CONSISTENCY is a prerequisite for native Landscape Redistribution as well as DDO. All necessary steps are described in SAP Note 1908075 (BW on SAP HANA: Table placement and landscape redistribution).

Best regards,

Kostas

[nicholas_chang]

Thanks for enlighten! much clearer now.

cheers,

Nicholas Chang

[manish_umarwadia2]

Hi Klaus,

Is this an accurate recap (not including the extended storage server concept)?

-PSA tables, Change log tables and write-optimized DSOs should be able to use the early unload option since there’s only one active partition “in-play” even now at 7.40 SP10.

-With 7.40 SP10, we should be able to “tag” one “regular” node as an extension node (Option 1 in the blog), although early unload might yield the same result

-With 7.50 SP01 we can use DT with advanced DSOs (aDSO) provided we range partition them

-With 7.50 SP05 range partitioning of aDSO will be made available using functional tools.

-Infocubes and classic DSOs can NOT use DT but could use early unload if they pertain to historical years.

-If SPOs use rolling time window option (Juergen Haupt's blog) then can older partitions use DT?

Since the bulk of the data is likely going to be contained in aDSOs with activation, seems like range partitioning is the only option. Not sure what happens if the activation/queue and activation impacts a warm partition.

Thanks,

Manish

[klaus_nagel]

Hi Manish,

the "extension nodes" concept takes the "early unload" option a big step further as it re-locates the data to a dedicated node, while the "early unload" data can reside on any node. So it is a significant difference with respect to load management and SLA management for "warm" data.

DT (HANA Dynamic Tiering) is a different technology. It covers similar data categories, but offers less flexibility and functionality compared to the "extension node" concept - at least for BW. Therefore we recommend customers to use "extension nodes".

SPOs are available only for InfoCubes and classic DSOs. Since both objects do not support DT (and not "extension nodes"), an SPO (or its partitions) can not use DT.

Wrt the aDSO activation please see the table in the blog.

Regards, Klaus

[Former Member]

I am  reading the blog and I have come across the mention about SPO based info cube and DSO's.

Are you saying DT won't work on SPO based objects?IF yes,how about extension nodes ?Will this new concept accommodate SPO based objects?

Thanks,
Swapna

[manish_umarwadia2]

See Klaus' reply:

"SPOs are available only for InfoCubes and classic DSOs. Since both objects do not support DT (and not "extension nodes"), an SPO (or its partitions) can not use DT."

SAP is recommending that we only use extension node concept for BW.

DT anyway addresses an entire table and not part of a table so would not be too useful for BW objects like aDSO.

[Former Member]

klaus.nagel - Is  warm data for Classic DSO's an upcoming feature?

[klaus_nagel]

Hi Krishna,

no, all our investment goes into the new (advanced) DSO. For classic DSOs only the write-optimized DSOs can be enabled for the Extension Nodes.

Regards, Klaus

[Former Member]

Thanks for the quick feedback!! What options are there to move from Classic DSO to ADSO when you have a downstream DWH which consumes lots of Sales, Finance, Warehouse, GTS etc. extractors as delta.

[rogerloe]

Hi

If a customer buys BW/4 HANA under Full Use License and deploys Extension Nodes - is the data stored on disk seen in the Extension Node as the GB to license - or is it just the actual RAM in the Extension node?

i.e. customer has a 1TB RAM Extension Node but has 2TB of data on disk under Extension Node

Regards

Roger

[former_member197235]

Hi There,

Does "HANA Extension Nodes for BW-on-HANA" support SAP HANA System Replication?

Thanks,

Amada

[richard_bremer]

Hi Roger,

please understand that we will not discuss licensing-related questions on SCN. You may want to refer to SAP Note 1704499 which explains license measurements based on memory utilization, also applicable to extension nodes.

Please check with your SAP account team for more details regarding licensing.

Best regards,

Richard

[tombo_larsen]

Hi Klaus

Do SAP have any update on the support for option 3 ("Special") using a speciel extension node?

One example for a two-node scale-out:  HANA-node has 4 TB RAM and extension-node has 2 TB RAM.

Best regards

Tom Bo

 

[shonalikellogg02]

HI Klaus,

 

i have similar question to Tom Bo and am interested in finding out if the principles of HA are available for Extension nodes.

Thanks.

 

Rgds,

Shonali

 

[Former Member]

Hi Klaus,

It's not clear to me the affect (if any) of sizing for the persistence layer. When SAP refers to the extension node max of 2*n*TB of data exactly how much storage data persistence would be required?  If it was a 2TB extension node, then does this mean a max of 2*2TB*3 needed for the data persistence storage space?  (If we are using are standard appliance nodes as extension node in this case, we probably don’t have the extra space needed since we would have sized a 2TB node with 6TB of data persistence, not 12TB.)

Regards,

Mark

 

[former_member623107]

Hi Klaus & Others,

We are exploring the option of using extension node on our BW on HANA system. We have tons of classic DSOs and Info Cubes that can't be categorized as "warm" as per this blog.

Please advise:

• Are there any tools provided by SAP that can convert DSOs/Info Cubes into Advanced DSOs, without impacting the data and dataflows and the Reports that were already built on top of these Info-Providers.