SELinux Troubleshooting - Nautobot Documentation

Skip to content

# SELinux Troubleshooting

When installing Nautobot for the first time on a Redhat-based Linux Distribution, SELinux may prevent the Nautobot stack from working properly. An example is SELinux preventing the HTTP daemon (NGINX, et al.) from communicating to the Django application stack on the backend.

## Determine if SELinux is the Culprit

An example of a broken application can be seen in the Nginx error logs below:

    sudo tail -f /var/log/nginx/error.log

Example output:

    2021/02/26 15:16:55 [crit] 67245#0: *494 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 47.221.167.40, server: nautobot.example.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "nautobot.example.com"
    2021/02/26 15:16:56 [crit] 67245#0: *494 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 47.221.167.40, server: nautobot.example.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8080/favicon.ico", host: "nautobot.example.com", referrer: "https://nautobot.example.com/"
    2021/02/26 15:16:58 [crit] 67245#0: *544 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 47.221.167.40, server: nautobot.example.com, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8080/", host: "nautobot.example.com"
    2021/02/26 15:16:58 [crit] 67245#0: *544 connect() to 127.0.0.1:8080 failed (13: Permission denied) while connecting to upstream, client: 47.221.167.40, server: nautobot.example.com, request: "GET /favicon.ico HTTP/1.1", upstream: "http://127.0.0.1:8080/favicon.ico", host: "nautobot.example.com", referrer: "https://nautobot.example.com/"

A quick way to verify that SELinux is preventing the application from working is to first, verify that SELinux is indeed `enabled` and operating in `enforcing` mode, and second, temporarily put SELinux in `permissive` mode. With SELinux in `permissive` mode, the application stack can be tested again. If the application starts working as expected, then SELinux is most likely the culprit.

    # sestatus | egrep 'SELinux status|Current mode'
    SELinux status:                 enabled
    Current mode:                   enforcing

To put SELinux in `permissive` mode, execute the `setenforce` command with the `0` flag.

    # setenforce 0

    # sestatus | egrep 'SELinux status|Current mode'
    SELinux status:                 enabled
    Current mode:                   permissive

With SELinux in `permissive` mode, test the application stack and ensure everything is working properly. If the application is working, put SELinux back into `enforcing` mode. This is done by executing the `setenforce` command with the `1` flag.

    # setenforce 1

    # sestatus | egrep 'SELinux status|Current mode'
    SELinux status:                 enabled
    Current mode:                   enforcing

## Troubleshoot SELinux

Troubleshooting SELinux in most instances is straightforward. Using the `sealert` command to parse `/var/log/audit/audit.log` is the fastest way to pin-point SELinux specific issues. In many cases, `sealert` will also provide guidance as to how to resolve the issue.

    # sealert -a /var/log/audit/audit.log
    100% done
    found 1 alerts in /var/log/audit/audit.log
    --------------------------------------------------------------------------------

    SELinux is preventing /usr/sbin/nginx from name_connect access on the tcp_socket port 8080.

    *****  Plugin connect_ports (85.9 confidence) suggests   *********************

    If you want to allow /usr/sbin/nginx to connect to network port 8080
    Then you need to modify the port type.
    Do
    # semanage port -a -t PORT_TYPE -p tcp 8080
        where PORT_TYPE is one of the following: dns_port_t, dnssec_port_t, kerberos_port_t, ocsp_port_t.

    *****  Plugin catchall_boolean (7.33 confidence) suggests   ******************

    If you want to allow httpd to can network connect
    Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.

    Do
    setsebool -P httpd_can_network_connect 1

    *****  Plugin catchall_boolean (7.33 confidence) suggests   ******************

    If you want to allow nis to enabled
    Then you must tell SELinux about this by enabling the 'nis_enabled' boolean.

    Do
    setsebool -P nis_enabled 1

    *****  Plugin catchall (1.35 confidence) suggests   **************************

    If you believe that nginx should be allowed name_connect access on the port 8080 tcp_socket by default.
    Then you should report this as a bug.
    You can generate a local policy module to allow this access.
    Do
    allow this access for now by executing:
    # ausearch -c 'nginx' --raw | audit2allow -M my-nginx
    # semodule -X 300 -i my-nginx.pp

    Additional Information:
    Source Context                system_u:system_r:httpd_t:s0
    Target Context                system_u:object_r:unreserved_port_t:s0
    Target Objects                port 8080 [ tcp_socket ]
    Source                        nginx
    Source Path                   /usr/sbin/nginx
    Port                          8080
    Host                          <Unknown>
    Source RPM Packages           nginx-1.14.1-9.module_el8.0.0+184+e34fea82.x86_64
    Target RPM Packages
    SELinux Policy RPM            selinux-policy-targeted-3.14.3-54.el8_3.2.noarch
    Local Policy RPM              selinux-policy-targeted-3.14.3-54.el8_3.2.noarch
    Selinux Enabled               True
    Policy Type                   targeted
    Enforcing Mode                Enforcing
    Host Name                     nautobot.example.com
    Platform                      Linux nautobot.example.com
                                  4.18.0-240.1.1.el8_3.x86_64 #1 SMP Thu Nov 19
                                  17:20:08 UTC 2020 x86_64 x86_64
    Alert Count                   5
    First Seen                    2021-02-26 15:16:55 UTC
    Last Seen                     2021-02-26 15:23:12 UTC
    Local ID                      b83bb817-85f6-4f5c-b6e0-eee3acc85504

    Raw Audit Messages
    type=AVC msg=audit(1614352992.209:585): avc:  denied  { name_connect } for  pid=67245 comm="nginx" dest=8080 scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:object_r:unreserved_port_t:s0 tclass=tcp_socket permissive=0

    type=SYSCALL msg=audit(1614352992.209:585): arch=x86_64 syscall=connect success=no exit=EACCES a0=12 a1=55d061477358 a2=10 a3=7ffc0c62296c items=0 ppid=67243 pid=67245 auid=4294967295 uid=988 gid=985 euid=988 suid=988 fsuid=988 egid=985 sgid=985 fsgid=985 tty=(none) ses=4294967295 comm=nginx exe=/usr/sbin/nginx subj=system_u:system_r:httpd_t:s0 key=(null)ARCH=x86_64 SYSCALL=connect AUID=unset UID=nginx GID=nginx EUID=nginx SUID=nginx FSUID=nginx EGID=nginx SGID=nginx FSGID=nginx

    Hash: nginx,httpd_t,unreserved_port_t,tcp_socket,name_connect

In the first few lines of the audit, `sealert` details what SELinux is blocking and provides some options to remedy the issue. Since Nginx is communicating with the Nautobot application via HTTP, the second option is the best option.

    SELinux is preventing /usr/sbin/nginx from name_connect access on the tcp_socket port 8080.

    *****  Plugin connect_ports (85.9 confidence) suggests   *********************

    If you want to allow /usr/sbin/nginx to connect to network port 8080
    Then you need to modify the port type.
    Do
    # semanage port -a -t PORT_TYPE -p tcp 8080
        where PORT_TYPE is one of the following: dns_port_t, dnssec_port_t, kerberos_port_t, ocsp_port_t.

    *****  Plugin catchall_boolean (7.33 confidence) suggests   ******************

    If you want to allow httpd to can network connect
    Then you must tell SELinux about this by enabling the 'httpd_can_network_connect' boolean.

    Do
    setsebool -P httpd_can_network_connect 1

Executing `setsebool -P httpd_can_network_connect 1` should remedy the issue. Verify this by executing the `setsebool` command, verify that SELinux is enabled and in `enforcing` mode via the `sestatus` command, and test the application stack for functionality.

The first curl command demonstrates the failure. Nginx responds with a HTTP response code of 502, indicating that it is unable to communicate with the Nautobot application. After executing the `setsebool` command, curl is used again to verify that Nginx is able to communicate with the Nautobot application. This is verified with the HTTP response code of 200.

    # curl -ik https://nautobot.example.com
    HTTP/1.1 502 Bad Gateway
    Server: nginx/1.14.1
    Date: Fri, 26 Feb 2021 15:41:22 GMT
    Content-Type: text/html
    Content-Length: 173
    Connection: keep-alive

    # sestatus
    SELinux status:                 enabled
    SELinuxfs mount:                /sys/fs/selinux
    SELinux root directory:         /etc/selinux
    Loaded policy name:             targeted
    Current mode:                   enforcing
    Mode from config file:          enforcing
    Policy MLS status:              enabled
    Policy deny_unknown status:     allowed
    Memory protection checking:     actual (secure)
    Max kernel policy version:      32

    # setsebool -P httpd_can_network_connect 1

    # curl -ik https://nautobot.example.com
    HTTP/1.1 200 OK
    Server: nginx/1.14.1
    Date: Fri, 26 Feb 2021 15:41:49 GMT
    Content-Type: text/html; charset=utf-8
    Content-Length: 18698
    Connection: keep-alive
    X-Content-Type-Options: nosniff
    Referrer-Policy: same-origin
    X-Frame-Options: DENY
    Vary: Cookie, Origin