12 Replies · Latest reply on Jul 13, 2017 8:07 PM by Craig Brown

    Inconistent Linux Discovery Results

      Hi there,

       

      I have a discovery running which uses the SSH, WMI, SNMP and DNS port probes. It gets inconsistent results for Linux classification even though I'm using the same SSH credentials each time it runs.

       

      Here is a successful classification with the CI being updated.

       

      And then the following day i get this for the same server with the same SSH credentials.

       

      The messages/warnings in the discovery log for the server are exactly the same for both of the above discoveries so I'm struggling to see what the difference between them is.

       

      Any ideas?

        • Re: Inconistent Linux Discovery Results
          David Ainsworth

          Hi Craig,

           

          The warning shown above is not really an issue (assuming these are not Amazon EC2 instances) because it is failing to find a file that only exists on EC2 instances. If you are not discovering EC2 instances, you can disable the probe by going to the following:

           

          Discovery Definition->Properties

           

          Then set the following property to false:

           

          glide.discovery.discover_aws_ec2_host_metadata   (When doing IP-based discovery against a given host, also run probes that retrieve AWS EC2 metadata.)

           

          This will remove the false warnings for these servers and then maybe make it easier to troubleshoot.

           

          What is interesting though is that the EC2 probe is only triggered during exploration which means it must have successfully got past classification and identification for this probe to trigger but you are getting could not classify message.

           

          Are there any other errors in the discovery log? Perhaps after setting that property, it will be easier to see if there are other issues.

           

          Regards,

           

          Dave

            • Re: Inconistent Linux Discovery Results
              Craig Brown

              Here are some of the other errors which are mostly down to the SUDO account not having enough permissions to run all the scripts. But i guess what my main question is about is why the CI is updated successfully one time and not the next when the exact same probes and sensors are being run against every discovery.

                • Re: Inconistent Linux Discovery Results
                  Craig Brown

                  Update. I got rid of some of the warnings by adding the "must_sudo" variable to some probes but I'm still getting errors for the MySQL and Postgres probes becuase they can't find files.

                   

                   

                  As a test I tried to remove the "PostgresSQL - Configuration"  and "MySQL - Configuration" probes from discovery completely using the below tick-box but I'm still seeing those probes being running. Is this a bug or am i doing something wrong?

                    • Re: Inconistent Linux Discovery Results
                      David Ainsworth

                      Hi Craig,

                       

                      The best way to do this is to disable the process classifier. So go to Discovery Definition->Processes and uncheck the active checkbox for which ever application instances you don’t wish to discover. In the case above, set the 'PostgreSQL Instance' to inactive. That will prevent the probes being triggered when the process is detected.

                       

                      I think the ‘used by’ filters the list of probes available when you go to a classier and click on Edit on the ‘Triggers Probes’ but if a probe is already in that list it might still get triggered even if used by discovery has later been unchecked.

                       

                      Regards,

                       

                      Dave

                        • Re: Inconistent Linux Discovery Results
                          David Ainsworth

                          Hi Craig,

                           

                          As for the inconsistent discovery results. I know of at least two ways in which this can happen:

                           

                          1. The credential record has the 'Applies to' set to specific MID servers. Therefore some MID servers cannot use the correct account and discovery results become inconsistent depending on which MID server in a cluster is used.
                          2. Possibly more likely (and more difficult to diagnose) is that the account is getting locked and then getting automatically unlocked after certain period of time (e.g. half an hour). This will mean that any probe triggered in that half hour will fail because of the locked account. Can you check with your linux admin if this is the case and see if there is any particular device where this is happening?

                           

                          Regards,


                          Dave

                           

                           

                          Edit: Just thought, are you seeing the Veritas Storage probe being triggered at all? This is only triggered on certain servers when a certain condition is met. Go to Discovery->ECC Queue and search for *vxvm in the name column.

                            • Re: Inconistent Linux Discovery Results
                              Craig Brown

                              David,

                               

                              Disabling the probes worked as you described, thank you. However even with no discovery warnings for a given Linux server I'm still getting "Active, couldn't classify". Except I'm getting all the time now.

                               

                              I only have one MID server so your comment 1 above doesn't apply here. Also the warning messages I'm getting after configuring "must_sudo" suggest that the account is successfully authenticating it's just that the probes can't find certain files (see below).

                               

                               

                              I'm more confused that ever.

                                • Re: Inconistent Linux Discovery Results
                                  David Ainsworth

                                  Hi Craig,

                                   

                                  Those warnings will come from MySQL and PostgreSQL application probes and won’t be affecting classification.

                                   

                                  Can you check the Most recent discovered field (last_discovered) on the servers? If this field is being updated then it will probably be updating the CI.

                                   

                                  Is there perhaps a SNMP classifier probe being triggered as well as the Unix Classify? This sometimes happens if there is a SNMP agent on the server and thinking if this will create the error you are seeing but only if you have changed the order of the port probes. If there is an SNMP classifier, you could use a behaviour to limit the discovery to SSH (to test this fixes the error) or you could set the following property to true:

                                   

                                  glide.discovery.ip_service_affinity

                                   

                                  After setting this property and discovering a server, it should recognise that SSH classification was successful and use this as a priority the next time the server is discovered.

                                   

                                  Regards,


                                  Dave