How do I debug OpenStack? - Ask OpenStack: Q&A Site for OpenStack Users and Developers
Description: I stumbled upon an error that I cannot decipher: where do I find OpenStack's logs and how do I debug the system?
Keywords: ,Ask, OpenStack,forum,community,cloud,iaas

First time here? Check out the FAQ!

English ▼
Hi there! Please sign in tags users badges help

| ALL UNANSWERED |  | Ask Your Question

8

# How do I debug OpenStack?

- faq
- gettingstarted
- howto

asked 2013-04-05 19:10:34 -0600

Anonymous

updated 2014-09-09 13:50:16 -0600

Santosh_Parihar
1 ●3 ●4 ●4

I stumbled upon an error that I cannot decipher: where do I find OpenStack's logs and how do I debug the system?

edit retag flag offensive close merge delete
add a comment

## 5 answers

Sort by » oldest newest most voted
5

answered 2013-05-08 00:06:44 -0600

fifieldt
4637 ●12 ●48 ●49 http://docs.openstack....

updated 2013-10-30 13:02:34 -0600

smaffulli
6898 ●36 ●67 ●100 http://maffulli.net/

Hi,

The OpenStack Operations guide covers this:

- http://docs.openstack.org/trunk/openstack-ops/content/lay_of_the_land.html
- http://docs.openstack.org/trunk/openstack-ops/content/maintenance.html
- http://docs.openstack.org/trunk/openstack-ops/content/network_troubleshooting.html
- http://docs.openstack.org/trunk/openstack-ops/content/logging_monitoring.html

edit flag offensive delete link more
add a comment
9

answered 2013-04-05 19:27:03 -0600

This post is a wiki. Anyone with karma >75 is welcome to improve it.

Debugging a piece of code is an activity as important as writing it. The importance of debugging is much more noticeable on large projects like OpenStack, which require both developers and common users contribute and report when something is not going well.

Here are some pointers to collect useful logs and debugging processes.

Logging

Looking through the logs can be tricky... not if you know how to use regular expressions!

The first thing to do when we see a strange behavior that is repeated with the same input data, is to look at logs.

Logs are files, usually plain text and arranged chronologically, that contain information about the code that is executed to perform an action.

Analyzing the information found there, you may be able to detect what the problem is or, at least, where the error occurs.

The location of the logs in the system may vary depending on the implementation, but in GNU/Linux they usually are stored in /var/log.

In OpenStack the location of these logs varies according how have you deployed it: using DevStack or by hand, whether with the distribution repository packages you are using or from source.

If you're using DevStack keep in mind that by default, DevStack doesn’t log. To enable this option you will need to add the following lines to the localrc file.

> LOGFILE=$DEST/stack.sh.log

– Name and location of the stack.sh script logs. Without this line the output of this script will appear in the terminal where you run it, without being stored in any location.

> SCREEN_LOGDIR=$DEST/logs/screen

– Location of the screen logs. DevStack runs OpenStack services under GNU Screen, thus providing the facility to view its current execution status. The logs simply store the output of these screens..

> LOGDAYS=1

– Number of days to be logged. The old logs are replaced with the latest runs.

$DEST is the location in which the logs will be located. You can set this value as you wish.

Screen logs are just a record of the screens initiated by DevStack but you can check them anytime during the execution. The information provided is very useful to debug any OpenStack service.

GNU Screen provides the same functions as the DEC VT100 terminal, plus other aggregates. If you have no experience with this terminal, make sure to write down the following shortcuts.

     Attach to a screen      screen -r stack
     Switch screens          ctrl+a <screen no.>  (e.g. ctrl+a 1)
     Scroll in a screen      ctrl+a esc and up/down arrows to scroll
     List screens        ctrl+a " and select the screen with up/down arrows
     Next screen         ctrl+a n
     Detach from the screen  ctrl+a d

Manual deployment

Here you will find most services logs in /var/log/service, where service is the name of the desired service. For example, in /var/log/keystone you will find Keystone logs.

In my experience, the only service that differs from this default location is Horizon. You can find ... (more)

edit flag offensive delete link more

## Comments

HI, are there any guide on debugging with eclipse and pydev with the latest branch. I have tried commented out eventlet.monkeypatch(os=False) and replaced it with eventlet.monkeypatch(all=False,socket=True,time=True,os=False) and added import sys;sys.path.append('path') but breakpoints are ignored

sak ( 2013-05-07 02:08:47 -0600 )edit

@sak thanks for asking. I believe yours would be a very good new question, more than a comment here. Do you mind posting it as a new question?

smaffulli ( 2013-05-07 12:05:23 -0600 )edit

cool. I have posted this as a new question: https://ask.openstack.org/question/815/how-do-i-debug-nova-service-with-eclipse-and-pydev/

sak ( 2013-05-08 03:05:22 -0600 )edit
add a comment
4

answered 2013-10-30 08:38:57 -0600

RomilGupta
814 ●6 ●16 ●29 http://ilearnstack.com/

Here is the link to debug openstack using eclipse

http://ilearnstack.com/2013/05/30/debug-openstack-code-local-remote-with-eclipse-and-pydev-plug-in/

edit flag offensive delete link more
add a comment
1

answered 2013-10-29 12:27:03 -0600

maurosr
11 ●1 ●2

updated 2013-10-30 13:03:31 -0600

smaffulli
6898 ●36 ●67 ●100 http://maffulli.net/

Just use pdb doesn't work fine with some projects like nova and or tempest, I'm not really sure, but I think it has something to do to how testr perform the tests run. Although there is valuable tip on testr page on the wiki https://wiki.openstack.org/wiki/Testr#Debugging_.28pdb.29_Tests

TL;DR: python -m testtools.run discover my.tests.regex

edit flag offensive delete link more
add a comment
1

answered 2013-10-01 04:29:30 -0600

unmesh-gurjar
1111 ●2 ●8 ●21

updated 2013-10-01 04:30:55 -0600

Please refer Grizzly docs http://docs.openstack.org/grizzly/openstack-ops/content/logging_monitoring.html

Compute specific docs here http://docs.openstack.org/grizzly/openstack-compute/admin/content/configuring-logging.html

edit flag offensive delete link more
add a comment
1

answered 2014-09-09 06:17:47 -0600

SGPJ
10656 ●59 ●116 ●154 https://in.linkedin.co...

You should be seeing logs under: /var/log/* dir under respective service.

edit flag offensive delete link more
add a comment
1

answered 2013-10-03 05:56:11 -0600

sngirame
368 ●25 ●33 ●39

Networking logs can be found at /var/log/quantum/

edit flag offensive delete link more
add a comment
0

answered 2014-09-15 02:41:36 -0600

Khayam
138 ●8 ●9 ●21 https://ask.openstack....

updated 2015-07-16 04:11:53 -0600

vincent-legoll
690 ●2 ●7 ●14 http://www.france-gril...

Check this guide. It seems good.

edit flag offensive delete link more
add a comment

## Your Answer

Please start posting anonymously - your entry will be published after you log in or create a new account.

Add Answer

[hide preview]

# Get to know Ask OpenStack

Resources for moderators

## Question Tools

3 followers

## Stats

Asked: 2013-04-05 19:10:34 -0600

Seen: 6,144 times

Last updated: Jul 16 '15

## Related questions

Recommended hardware to learn?

How Many Physical Servers and with what Resources?

How can I connect to my VM from outside?

Why do I need to provide contact information when signing the CLA?

What should I put into Schedule A inside CCLA?

Why is my VM running slow

Where can I find the OpenStack Roadmap?

nova-novncproxy service not installed

how can I get start with openstack [closed]

Asking for an advice for a career path. [closed]

( 2016-11-08 09:24:23 -0600 )edit
none