Skip to content

GitLab

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
fleur
fleur
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 51
    • Issues 51
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
  • Operations
    • Operations
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • Repository
    • Value Stream
  • Wiki
    • Wiki
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Commits
  • Issue Boards
  • fleur
  • fleurfleur
  • Issues
  • #498

Closed
Open
Opened Sep 01, 2020 by Henning Janssen@janssenDeveloper

Weird memory usage with second variation SOC and ev parallelization

Summary

I often encounter calculations on iffslurm, where Fleur runs out of memory after 10 or 20 iterations. I discovered that this happens more often with second variation SOC, so I decided to make a few tests. I used the input below with second variation SOC with and without eigenvalue parallelization NdCo5_evpara.job NdCo5_purekpara.job and looked at the memory usage by patching the data for a console output from a run with -debugtime together. I also performed a run with first variation SOC and eigenvalue parallelization.

In these files I plot the memory usage versus the index of occurence in the console output (not time !!!) for the three cases

mem_eigenvaluepara.pdf mem_kpara.pdf mem_nocoeigenvaluepara.pdf

You can clearly make out the Diagonalization and cdngen parts of the timers as the periodic oscillations in the usage (I think they have roughly the same number of timerstarts/stops)

As you can see in mem_eigenvaluepara.pdf the memory usage increases dramatically over multiple iterations. It more than doubles in ~10 iterations. It can be split up into two contributions, which I confirmed with the pure k-parallelization run and the first variation test with ev parallelization

  • With ev parallelization some tasks use drastically more memory and the memory usage increases in the diagonalization part
  • With second variation SOC there is an increase in memory usage in the cdngen part (you can clearly see that this is not present in mem_nocoeigenvaluepara.pdf)

Note: I had to reduce the parameters a bit for the noco run. I can also perform a noco run with pure k-parallelization. This should not have any of these problems in the issue

Input and output file:

inp.xml

kpts.xml

sym.xml

Compute environment

iffslurm

I'm not sure where this comes from, but in my opinion the memory usage should not increase this much between iterations

Edited Sep 01, 2020 by Henning Janssen
Assignee
Assign to
None
Milestone
None
Assign milestone
Time tracking
None
Due date
None
Reference: fleur/fleur#498