{"id":171,"date":"2012-09-03T15:17:19","date_gmt":"2012-09-03T15:17:19","guid":{"rendered":"http:\/\/www.andreagirardi.it\/blog\/?p=171"},"modified":"2014-01-22T14:25:20","modified_gmt":"2014-01-22T14:25:20","slug":"linux-mow-much-ram-is-used-per-program","status":"publish","type":"post","link":"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/","title":{"rendered":"Linux: mow much RAM is used per program"},"content":{"rendered":"<p>The script is a very helpful to show how much RAM is used by single process. This script is written by P\u00e1draig Brady, He can be contacted at P@draigBrady.com. The script itself follows:<\/p>\n<pre lang=\"bash\">#!\/usr\/bin\/env python\r\n\r\n# Try to determine how much RAM is currently being used per program.\r\n# Note the per program, not per process. So for example this script\r\n# will report mem used by all httpd process together. In detail it reports:\r\n# sum(all RSS for process instances) + max(shared mem for any process instance)\r\n#\r\n# The shared calculation below will factor out shared text and\r\n# libs etc. within a program, but not between programs. So there\r\n# will always be some overestimation. This will be the same for\r\n# all processes that just use libc for e.g. but more for others\r\n# that use larger shared libs like gnome, kde etc.\r\n\r\n# Author: P@draigBrady.com\r\n\r\n# V1.0      06 Jul 2005    Initial release\r\n# V1.1      11 Aug 2006    root permission required for accuracy\r\n# V1.2      08 Nov 2006    Add total to output\r\n#                          Use KiB,MiB,... for units rather than K,M,...\r\n# V1.3      22 Nov 2006    Ignore shared col from \/proc\/$pid\/statm for\r\n#                          2.6 kernels up to and including 2.6.9.\r\n#                          There it represented the total file backed extent\r\n# V1.4      23 Nov 2006    Remove total from output as it's meaningless\r\n#                          (the shared values overlap with other programs).\r\n#                          Display the shared column. This extra info is\r\n#                          useful, especially as it overlaps between programs.\r\n# V1.5      26 Mar 2007    Remove redundant recursion from human()\r\n# V1.6      05 Jun 2007    Also report number of processes with a given name.\r\n#                          Patch from riccardo.murri@gmail.com\r\n\r\n# Notes:\r\n#\r\n# All interpreted programs where the interpreter is started\r\n# by the shell or with env, will be merged to the interpreter\r\n# (as that's what's given to exec). For e.g. all python programs\r\n# starting with \"#!\/usr\/bin\/env python\" will be grouped under python.\r\n# You can change this by changing comm= to args= below but that will\r\n# have the undesirable affect of splitting up programs started with\r\n# differing parameters (for e.g. mingetty tty[1-6]).\r\n#\r\n# For 2.6 kernels up to and including 2.6.13 and later 2.4 redhat kernels\r\n# (rmap vm without smaps) it can not be accurately determined how many pages\r\n# are shared between processes in general or within a program in our case:\r\n# http:\/\/lkml.org\/lkml\/2005\/7\/6\/250\r\n# A warning is printed if overestimation is possible.\r\n# In addition for 2.6 kernels up to 2.6.9 inclusive, the shared\r\n# value in \/proc\/$pid\/statm is the total file-backed extent of a process.\r\n# We ignore that, introducing more overestimation, again printing a warning.\r\n#\r\n# I don't take account of memory allocated for a program\r\n# by other programs. For e.g. memory used in the X server for\r\n# a program could be determined, but is not.\r\n#\r\n# This script assumes threads are already merged by ps\r\n\r\n# TODO:\r\n#\r\n# use ps just to enumerate the pids and names\r\n# so as to remove the race between reading rss and shared values\r\n\r\nimport sys, os, string\r\n\r\nif os.geteuid() != 0:\r\n    sys.stderr.write(\"Sorry, root permission required.\\n\");\r\n    sys.exit(1)\r\n\r\nPAGESIZE=os.sysconf(\"SC_PAGE_SIZE\")\/1024 #KiB\r\nour_pid=os.getpid()\r\n\r\n#(major,minor,release)\r\ndef kernel_ver():\r\n    kv=open(\"\/proc\/sys\/kernel\/osrelease\").readline().split(\".\")[:3]\r\n    for char in \"-_\":\r\n        kv[2]=kv[2].split(char)[0]\r\n    return (int(kv[0]), int(kv[1]), int(kv[2]))\r\n\r\nkv=kernel_ver()\r\n\r\ndef getShared(pid):\r\n    if os.path.exists(\"\/proc\/\"+str(pid)+\"\/smaps\"):\r\n        shared_lines=[line\r\n                      for line in open(\"\/proc\/\"+str(pid)+\"\/smaps\").readlines()\r\n                      if line.find(\"Shared\")!=-1]\r\n        return sum([int(line.split()[1]) for line in shared_lines])\r\n    elif (2,6,1) <= kv <= (2,6,9):\r\n        return 0 #lots of overestimation, but what can we do?\r\n    else:\r\n        return int(open(\"\/proc\/\"+str(pid)+\"\/statm\").readline().split()[2])*PAGESIZE\r\n\r\ncmds={}\r\nshareds={}\r\ncount={}\r\nfor line in os.popen(\"ps -e -o rss=,pid=,comm=\").readlines():\r\n    size, pid, cmd = map(string.strip,line.strip().split(None,2))\r\n    if int(pid) == our_pid:\r\n        continue #no point counting this process\r\n    try:\r\n        shared=getShared(pid)\r\n    except:\r\n        continue #ps gone away\r\n    if shareds.get(cmd):\r\n        if shareds[cmd] < shared:\r\n            shareds[cmd]=shared\r\n    else:\r\n        shareds[cmd]=shared\r\n    #Note shared is always a subset of rss (trs is not always)\r\n    cmds[cmd]=cmds.setdefault(cmd,0)+int(size)-shared\r\n    if count.has_key(cmd):\r\n      count[cmd] += 1\r\n    else:\r\n      count[cmd] = 1\r\n\r\n#Add max shared mem for each program\r\nfor cmd in cmds.keys():\r\n    cmds[cmd]=cmds[cmd]+shareds[cmd]\r\n\r\nsort_list = cmds.items()\r\nsort_list.sort(lambda x,y:cmp(x[1],y[1]))\r\nsort_list=filter(lambda x:x[1],sort_list) #get rid of zero sized processes (kernel threads)\r\n\r\n#The following matches \"du -h\" output\r\n#see also human.py\r\ndef human(num, power=\"Ki\"):\r\n    powers=[\"Ki\",\"Mi\",\"Gi\",\"Ti\"]\r\n    while num >= 1000: #4 digits\r\n        num \/= 1024.0\r\n        power=powers[powers.index(power)+1]\r\n    return \"%.1f %s\" % (num,power)\r\n\r\ndef cmd_with_count(cmd, count):\r\n    if count>1:\r\n      return \"%s (%u)\" % (cmd, count)\r\n    else:\r\n      return cmd\r\nprint \" Private  +  Shared  =  RAM used\\tProgram \\n\"\r\nfor cmd in sort_list:\r\n    print \"%8sB + %8sB = %8sB\\t%s\" % (human(cmd[1]-shareds[cmd[0]]), human(shareds[cmd[0]]), human(cmd[1]),\r\n                                      cmd_with_count(cmd[0], count[cmd[0]]))\r\nprint \"\\n Private  +  Shared  =  RAM used\\tProgram \\n\"\r\n\r\n#Warn of possible inaccuracies\r\n#1 = accurate\r\n#0 = some shared mem not reported\r\n#-1= all shared mem not reported\r\ndef shared_val_accurate():\r\n    \"\"\"http:\/\/wiki.apache.org\/spamassassin\/TopSharedMemoryBug\"\"\"\r\n    if kv[:2] == (2,4):\r\n        if open(\"\/proc\/meminfo\").read().find(\"Inact_\") == -1:\r\n            return 1\r\n        return 0\r\n    elif kv[:2] == (2,6):\r\n        if os.path.exists(\"\/proc\/\"+str(os.getpid())+\"\/smaps\"):\r\n            return 1\r\n        if (2,6,1) <= kv <= (2,6,9):\r\n            return -1\r\n        return 0\r\n    else:\r\n        return 1\r\n\r\nvm_accuracy = shared_val_accurate()\r\nif vm_accuracy == -1:\r\n    sys.stderr.write(\"Warning: Shared memory is not reported by this system.\\n\")\r\n    sys.stderr.write(\"Values reported will be too large.\\n\")\r\nelif vm_accuracy == 0:\r\n    sys.stderr.write(\"Warning: Shared memory is not reported accurately by this system.\\n\")\r\n    sys.stderr.write(\"Values reported could be too large.\\n\")<\/pre>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-linkedin\"><a rel=\"nofollow\" data-shared=\"sharing-linkedin-171\" class=\"share-linkedin sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\"><span>LinkedIn<\/span><\/a><\/li><li class=\"share-reddit\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-reddit sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=reddit\" target=\"_blank\" title=\"Click to share on Reddit\"><span>Reddit<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow\" data-shared=\"sharing-twitter-171\" class=\"share-twitter sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\"><span>Twitter<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow\" data-shared=\"sharing-facebook-171\" class=\"share-facebook sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\"><span>Facebook<\/span><\/a><\/li><li class=\"share-skype\"><a rel=\"nofollow\" data-shared=\"sharing-skype-171\" class=\"share-skype sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=skype\" target=\"_blank\" title=\"Share on Skype\"><span>Skype<\/span><\/a><\/li><li class=\"share-pocket\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-pocket sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=pocket\" target=\"_blank\" title=\"Click to share on Pocket\"><span>Pocket<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"excerpt":{"rendered":"<p>The script is a very helpful to show how much RAM is used by single process. This script is written by P\u00e1draig Brady, He can be contacted at P@draigBrady.com. The script itself follows: #!\/usr\/bin\/env python # Try to determine how much RAM is currently being used per program. # Note the per program, not per [&hellip;]<\/p>\n<div class=\"sharedaddy sd-sharing-enabled\"><div class=\"robots-nocontent sd-block sd-social sd-social-icon-text sd-sharing\"><h3 class=\"sd-title\">Share this:<\/h3><div class=\"sd-content\"><ul><li class=\"share-linkedin\"><a rel=\"nofollow\" data-shared=\"sharing-linkedin-171\" class=\"share-linkedin sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=linkedin\" target=\"_blank\" title=\"Click to share on LinkedIn\"><span>LinkedIn<\/span><\/a><\/li><li class=\"share-reddit\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-reddit sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=reddit\" target=\"_blank\" title=\"Click to share on Reddit\"><span>Reddit<\/span><\/a><\/li><li class=\"share-twitter\"><a rel=\"nofollow\" data-shared=\"sharing-twitter-171\" class=\"share-twitter sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=twitter\" target=\"_blank\" title=\"Click to share on Twitter\"><span>Twitter<\/span><\/a><\/li><li class=\"share-facebook\"><a rel=\"nofollow\" data-shared=\"sharing-facebook-171\" class=\"share-facebook sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=facebook\" target=\"_blank\" title=\"Click to share on Facebook\"><span>Facebook<\/span><\/a><\/li><li class=\"share-skype\"><a rel=\"nofollow\" data-shared=\"sharing-skype-171\" class=\"share-skype sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=skype\" target=\"_blank\" title=\"Share on Skype\"><span>Skype<\/span><\/a><\/li><li class=\"share-pocket\"><a rel=\"nofollow\" data-shared=\"\" class=\"share-pocket sd-button share-icon\" href=\"https:\/\/www.andreagirardi.it\/blog\/linux-mow-much-ram-is-used-per-program\/?share=pocket\" target=\"_blank\" title=\"Click to share on Pocket\"><span>Pocket<\/span><\/a><\/li><li class=\"share-end\"><\/li><\/ul><\/div><\/div><\/div>","protected":false},"author":2,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"footnotes":""},"categories":[14],"tags":[27,25,26],"_links":{"self":[{"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/posts\/171"}],"collection":[{"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/users\/2"}],"replies":[{"embeddable":true,"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/comments?post=171"}],"version-history":[{"count":4,"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/posts\/171\/revisions"}],"predecessor-version":[{"id":373,"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/posts\/171\/revisions\/373"}],"wp:attachment":[{"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/media?parent=171"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/categories?post=171"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.andreagirardi.it\/blog\/wp-json\/wp\/v2\/tags?post=171"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}