Configure concurrent import and parallel loading for Nokia Altiplano
Improve the efficiency of large data imports from Nokia Altiplano by configuring concurrent imports and enabling parallel loading. This configuration allows the platform to run multiple data import and transformation jobs simultaneously.
Antes de Iniciar
Role required: TSOM Visibility admin
Por Que e Quando Desempenhar Esta Tarefa
Concurrent import and parallel loading improve performance by dividing large datasets into smaller partitions, allowing multiple transformation and collection jobs to run in parallel. This setup is useful when importing large volumes of equipment, logical connections, and network topology data during bulk or filtered discoveries.
Use the following guidance to optimize performance:
- If transformation is slow: Enable Concurrent import in the import schedule.
- If data collection is slow: In addition to enabling concurrent import, configure Parallel loading in the data source and update the system property.
Nota:
Parallel loading works in addition to concurrent import. It does not replace it.
Procedimento
- Navigate to .
-
Configure concurrent import or concurrent import
To Do the following To enable concurrent import - Navigate to the Import Schedule for the discovery job.
- Select the Concurrent import checkbox.
- Set the Partition method to Custom size.
- In the Partition sizee field, enter the number of records per partition (for example, 1000).
Nota:The system splits the dataset into import sets based on the partition size. Each import set is processed in parallel, improving the speed of data transformation.Configure parallel loading in the data source - Navigate to the corresponding Data Source record.
- Select the Parallel loading checkbox.
- Navigate to Nokia Altiplano > Properties.
- Open the system property sn_sgc_altiplano.parallel_number_of_data_source_jobs
- Set the value to the desired number of parallel jobs (for example,
3).Nota:By default, the number of data source jobs is set to1. Increasing this value allows the platform to execute multiple data collection jobs concurrently.