[완료] purify 사용하면 성공... 빼면 실패?
어느순간부터 발생하는 core dump를 확인하고자 솔라리스 시스템에 purify 평가판을 설치했습니다.
core dump 내용은 다음과 같았는데요...
#0 0xfedc7598 in realfree () from /usr/lib/libc.so.1
#1 0xfedc7e64 in cleanfree () from /usr/lib/libc.so.1
#2 0xfedc6f90 in _malloc_unlocked () from /usr/lib/libc.so.1
purify를 사용하여 확인하고자 실행을 시켰더니...
signal : 11을 발생하며 죽어나던 시스템이 잘 돌아갑니다... ㅡㅡ;;
purify 옵션을 빼면 여전히 죽어나구요.
purify 를 이용해서 log를 남기도록 셋팅하였는데...
해당 로그를 살펴보면 대강 아래의 내용과 같습니다.
하지만 죽지를 않으니 대체 어디서 어떻게 문제가 발생되었는지... 가장 큰 원인이 무엇이었는지 모르겠네요...
메모리 릭에 관한 메시지도 없고...
그저 예전 core 파일을 보았을때 "in realfree () from /usr/lib/libc.so.1" 구문이 잡혀나와서...
purify에서 잡아준 "_so_send+0x18 [libc.so.1 pc=0xfea7f238]" 구문이 눈에 보입니다.
** purify 옵션을 주고 make 했을때 왜 이상없이 돌아가는지 purify 옵션을 빼면 왜 죽어나는지 무척 궁금합니다.
**** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048 at Tue Nov 4 15:29:58 2008) * Purify 7.0 Solaris 2 (32-bit) (C) Copyright IBM Corporation. 1992, 2006 All Rights Reserved. * For contact information type: "purify -help" * Options settings: -output-limit=10000000 -messages=first -chain-length=20 \ -show-pc-offset -show-pc -log-file=/user1/prmgwv/test/mylog.%v.%p -g++=yes \ -purify -purify-home=/program/Rational/releases/purify.sol.7.0 \ -gcc3_path=/usr/local/bin/g++ \ -cache-dir=/program/Rational/releases/purify.sol.7.0/cache * License successfully checked out. * Command-line: GviScheduler -b -10047 -s GviScheduler -d -1 **** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048) **** Process 10051 about to exec /bin/sh as "sh". **** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048) **** UMR: Uninitialized memory read: * This is occurring while in: _so_send+0x18 [libc.so.1 pc=0xfea7f238] _tmax_writen+0x2c [libsvrucs.so pc=0xff25e770] _send_csmsg_to_clh+0x4fc [libsvrucs.so pc=0xff254454] tpcall+0x7e0 [libsvrucs.so pc=0xff2371d0] CTmaxApi::TPCall(CTmaxBuf&, char const*)+0x190 [libUtils.a pc=0xb06cc] CTmaxApi::TPCall(CTmaxBuf&, std::string<char, std::char_traits<char>, std::allocator<char>>)+0x54 [libUtils.a pc=0xb0514] CviServiceDomain::GetConfigServiceName(int)+0x190 [viServiceDomain.o pc=0x91564] CviScheduler::GetConfigServiceName()+0xd4 [viScheduler.o pc=0x8f904] CviScheduler::InitService()+0x24 [viScheduler.o pc=0x8f7f8] CviScheduler::RunScheduler()+0x38 [viScheduler.o pc=0x8fc64] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * Reading 144 bytes from 0x112df8 (misaligned) in the heap (6 bytes at 0x112e72 uninit). * Address 0x112df8 is at the beginning of a malloc'd block of 144 bytes. * This block was allocated from: malloc+0x4f4 [rtlib.o pc=0x4eab0] _hton_data_fdl+0xbc [libsvrucs.so pc=0xff288a4c] _send_csmsg_to_clh+0x188 [libsvrucs.so pc=0xff2540e0] tpcall+0x7e0 [libsvrucs.so pc=0xff2371d0] CTmaxApi::TPCall(CTmaxBuf&, char const*)+0x190 [libUtils.a pc=0xb06cc] CTmaxApi::TPCall(CTmaxBuf&, std::string<char, std::char_traits<char>, std::allocator<char>>)+0x54 [libUtils.a pc=0xb0514] CviServiceDomain::GetConfigServiceName(int)+0x190 [viServiceDomain.o pc=0x91564] CviScheduler::GetConfigServiceName()+0xd4 [viScheduler.o pc=0x8f904] CviScheduler::InitService()+0x24 [viScheduler.o pc=0x8f7f8] CviScheduler::RunScheduler()+0x38 [viScheduler.o pc=0x8fc64] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] **** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048) **** ABR: Array bounds read: * This is occurring while in: memcmp+0x44 [rtlib.o pc=0x68c10] _get_realloc_symbol+0xe64 [mib.c:3867 pc=0xfefa092c] netsnmp_sprint_realloc_objid_tree+0xf8 [mib.c:2962 pc=0xfef9cc84] sprint_realloc_objid+0x38 [mib.c:3075 pc=0xfef9d150] CVarConvert::OidToString(unsigned long const*, unsigned)+0x1ac [libSnmpApi.a pc=0xab38c] CSnmpApi::AddVariable(unsigned long const*, unsigned, variable_list*)+0xac [libSnmpApi.a pc=0xa6584] CSnmpApi::SnmpBulkWalk(char const*, long)+0x490 [libSnmpApi.a pc=0xa5f70] CvpnIfCsc::GetProtocol()+0x58 [libmVPN.a pc=0x99734] CvpnIfCsc::GetData(CSnmpApi&)+0x90 [libmVPN.a pc=0x996b8] CviSchedule::GetVrfList(viTargetInfo_t*, CvpnIfCsc*)+0x20c [viSchedule.o pc=0x8d374] CviSchedule::PreparePoller(long, int)+0x160 [viSchedule.o pc=0x8cab0] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * Reading 10 bytes from 0x111c20 in the heap (6 bytes at 0x111c24 illegal). * Address 0x111c20 is at the beginning of a malloc'd block of 4 bytes. * This block was allocated from: malloc+0x4f4 [rtlib.o pc=0x4eab0] strdup+0x14 [libc.so.1 pc=0xfe9fb02c] init_tree_roots+0x284 [parse.c:1155 pc=0xfefa8698] _init_snmp+0x68 [snmp_api.c:624 pc=0xfefb5290] init_snmp+0x58 [snmp_api.c:787 pc=0xfefb5890] CSnmpApi::Init()+0x38 [libSnmpApi.a pc=0xa2a70] CSnmpApi::CSnmpApi()+0xc8 [libSnmpApi.a pc=0xa2698] CviSchedule::GetVrfList(viTargetInfo_t*, CvpnIfCsc*)+0x1a0 [viSchedule.o pc=0x8d308] CviSchedule::PreparePoller(long, int)+0x160 [viSchedule.o pc=0x8cab0] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] **** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048) **** FMR: Free memory read: * This is occurring while in: std::string<char, std::char_traits<char>, std::allocator<char>>::~string<char, std::char_traits<char>, std::allocator<char>>()+0x1c [basic_string.h:261 pc=0xfecdce9c] vpninterface_t::~vpninterface_t()+0x24 [libmVPN.a pc=0x98bd0] CvpnIfCsc::~CvpnIfCsc()+0x124 [libmVPN.a pc=0x9912c] CviSchedule::PreparePoller(long, int)+0x378 [viSchedule.o pc=0x8ccc8] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * Reading 4 bytes from 0x275ff0 in the heap. * Address 0x275ff0 is 177 bytes past end of a freed block at 0x275f38 of 8 bytes. * This block was allocated from: malloc+0x4f4 [rtlib.o pc=0x4eab0] operator new(unsigned)+0x1c [new_op.cc:48 pc=0xfed14ad4] operator new(unsigned)+0xc [rtlib.o pc=0x878c0] CvpnIfCsc::AddData(std::string<char, std::char_traits<char>, std::allocator<char>>, int)+0x34 [libmVPN.a pc=0x99508] CvpnIfCsc::GetProtocol()+0x3ac [libmVPN.a pc=0x99a88] CvpnIfCsc::GetData(CSnmpApi&)+0x90 [libmVPN.a pc=0x996b8] CviSchedule::GetVrfList(viTargetInfo_t*, CvpnIfCsc*)+0x20c [viSchedule.o pc=0x8d374] CviSchedule::PreparePoller(long, int)+0x160 [viSchedule.o pc=0x8cab0] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * There have been 172 frees since this block was freed. **** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048) **** FMW: Free memory write: * This is occurring while in: __gnu_cxx::__exchange_and_add(int volatile*, int)+0x58 [atomicity.cc:97 pc=0xfed0d1e8] std::string<char, std::char_traits<char>, std::allocator<char>>::~string<char, std::char_traits<char>, std::allocator<char>>()+0x40 [basic_string.h:217 pc=0xfecdcec0] vpninterface_t::~vpninterface_t()+0x24 [libmVPN.a pc=0x98bd0] CvpnIfCsc::~CvpnIfCsc()+0x124 [libmVPN.a pc=0x9912c] CviSchedule::PreparePoller(long, int)+0x378 [viSchedule.o pc=0x8ccc8] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * Writing 4 bytes to 0x276e30 in the heap. * Address 0x276e30 is 8 bytes into a freed block at 0x276e28 of 16 bytes. * This block was allocated from: malloc+0x4f4 [rtlib.o pc=0x4eab0] operator new(unsigned)+0x1c [new_op.cc:48 pc=0xfed14ad4] operator new(unsigned)+0xc [rtlib.o pc=0x878c0] std::string<char, std::char_traits<char>, std::allocator<char>>::_Rep::_S_create(unsigned, unsigned, std::allocator<char> const&)+0x94 [new_allocator.h:81 pc=0xfecdc55c] std::string<char, std::char_traits<char>, std::allocator<char>>::_M_mutate(unsigned, unsigned, unsigned)+0x78 [basic_string.h:578 pc=0xfecdd000] std::string<char, std::char_traits<char>, std::allocator<char>>::push_back(char)+0x60 [basic_string.h:1360 pc=0xfecde618] COidIndex::GetString(int)+0x318 [libSnmpApi.a pc=0xa8104] CvpnIfCsc::GetProtocol()+0x344 [libmVPN.a pc=0x99a20] CvpnIfCsc::GetData(CSnmpApi&)+0x90 [libmVPN.a pc=0x996b8] CviSchedule::GetVrfList(viTargetInfo_t*, CvpnIfCsc*)+0x20c [viSchedule.o pc=0x8d374] CviSchedule::PreparePoller(long, int)+0x160 [viSchedule.o pc=0x8cab0] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * There have been 285 frees since this block was freed. **** Purify instrumented /user1/prmgwv/tmax/appbin/GviScheduler (pid 10048) **** FUM: Freeing unallocated memory: * This is occurring while in: free+0x324 [rtlib.o pc=0x4ee00] _ZdLpV+0x10 [libstdc++.so.6 pc=0xfed12800] operator delete(*)+0x70 [rtlib.o pc=0x8799c] std::string<char, std::char_traits<char>, std::allocator<char>>::_Rep::_M_destroy(std::allocator<char> const&)+0x30 [new_allocator.h:86 pc=0xfecdcad0] std::string<char, std::char_traits<char>, std::allocator<char>>::~string<char, std::char_traits<char>, std::allocator<char>>()+0x54 [basic_string.h:218 pc=0xfecdced4] vpninterface_t::~vpninterface_t()+0x24 [libmVPN.a pc=0x98bd0] CvpnIfCsc::~CvpnIfCsc()+0x124 [libmVPN.a pc=0x9912c] CviSchedule::PreparePoller(long, int)+0x378 [viSchedule.o pc=0x8ccc8] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * Attempting to free block at 0x276e28 already freed. * This block was allocated from: malloc+0x4f4 [rtlib.o pc=0x4eab0] operator new(unsigned)+0x1c [new_op.cc:48 pc=0xfed14ad4] operator new(unsigned)+0xc [rtlib.o pc=0x878c0] std::string<char, std::char_traits<char>, std::allocator<char>>::_Rep::_S_create(unsigned, unsigned, std::allocator<char> const&)+0x94 [new_allocator.h:81 pc=0xfecdc55c] std::string<char, std::char_traits<char>, std::allocator<char>>::_M_mutate(unsigned, unsigned, unsigned)+0x78 [basic_string.h:578 pc=0xfecdd000] std::string<char, std::char_traits<char>, std::allocator<char>>::push_back(char)+0x60 [basic_string.h:1360 pc=0xfecde618] COidIndex::GetString(int)+0x318 [libSnmpApi.a pc=0xa8104] CvpnIfCsc::GetProtocol()+0x344 [libmVPN.a pc=0x99a20] CvpnIfCsc::GetData(CSnmpApi&)+0x90 [libmVPN.a pc=0x996b8] CviSchedule::GetVrfList(viTargetInfo_t*, CvpnIfCsc*)+0x20c [viSchedule.o pc=0x8d374] CviSchedule::PreparePoller(long, int)+0x160 [viSchedule.o pc=0x8cab0] CviSchedule::RunSchedule(int)+0x5c [viSchedule.o pc=0x8c8d0] CviServiceDomain::RunSchedule(int)+0xb0 [viServiceDomain.o pc=0x9132c] CviScheduler::RunSchedule()+0xd4 [viScheduler.o pc=0x8fb34] CviScheduler::RunScheduler()+0x1f0 [viScheduler.o pc=0x8fe1c] usermain+0x44c [GviScheduler.o pc=0x92b88] _tmax_main+0x9d8 [libsvrucs.so pc=0xff20cd78] _start+0x80 [crt1.o pc=0x44860] * There have been 286 frees since this block was freed.
Hint
Linux/Unix Purify 는 instrumentation based 입니다. Windows 버젼은 아님.
uni2u님께서 겪고 계신 상황이 instrumentation based 의 문제점이라고 할수 있죠.
다시말해 내가 어떤 문제로 인해서 이 툴(purify)을 사용하는데 이 툴이 컴파일레벨에서 자기의 코드를 조금 심죠.
이 때문에 문제가 재현이 안되는 상황입니다. 혹은 purify가 malloc()같은 함수들을 래핑하고 있어서 문제가 발생하지 않을수 도 있습니다.
아마도 죽지는 않지만 purify 리포트를 보시면 빨간느낌표가 나타날겁니다. ^^;
ps. 평가판에 제한이 없는지도 확인해보심이 좋을듯 싶군요. purify 보다 valgrind 를 추천하고 싶군요. 솔라리스라면 DTrace 를 써보심도...^^;
그리고, IBM DeveloperWorks 에 보시면 Purify 튜토리얼도 있습니다. GDB 연동까지..므흣~
Hello World.
댓글 달기