# encoding: utf-8
# lintest/main.py
"""
lintest is an automation test framework developed using python3 + selenium + appium, which supports the automation of API & UI & Mobile.
What can lintest do?
    - support UI, API, Mobile(IOS & Android) Automation
    - run test case concurrently or one-by-one.
    - auto re-run failed testcases
    - auto generate the html report(under project/output)
    - can set timeout for each testcase, if not then will use the settings.TESTCASE_TIMEOUT as default
    - support 6 ways to execute testcases(see below details)
    - cross-platform (support Mac OS X, Linux, Windows)
    - support DataSet（CSV)
    - support set Authentication(token) for UI testcase
    - for UI testcase, Chromedriver will be auto installed by the Chrome's version
    - support save execution logs into testdb, should work with lintest-dashboard (show the execution_logs summary & details & histories)
    - support multiple tag names for one testcase (easy to group testcases)
    - support set priority for testcase (easy to group testcases, another way)

to use lintest framework, your project must has 2 configuration files(will be auto generated if not exists):
    1. settings/__init__.py is used to configure below items for the lintest
        - Environment (test environment: UAT, QA, LIVE ...)
        - TESTCASE_TIMEOUT (after Testcase_Timeout seconds, if the testcase's execution still not complete will throw TimeoutException)
        - RERUN_FLAG (control rerun failed testcases. Default is False)
        - DEBUG_RUN (keep the browser active after execution done. Default value is False)
        - Show_All_Chrome_Driver_Logs (whether show all the webdriver's logs, Default value is False)
        - QUEUE_SIZE (set queue_size as 8 means that there are at most 8 test cases can be executed concurrently)
        - SHOW_LOG_IN_CONSOLE = True ( whether show execution logs, Default value is True)
        - SAVE_LOG_TO_DB = False (save execution logs into testdb, should work with lintest-dashboard)
        - generate_xunit_result = False (save xunit report, default value is False, used for TestLink)

    2. settings/testcase_tags.py
        class Tags is used to define all the testcase's tags.
        set the tag for each testcase (tag can be any non-blank string, you can identify it follow your business.
            eg."smoke", "nightly", "ignore" .etc)
            Note:
                if one testcase's tag contains "ignore", then it will only be added into ignore_testcase_list, even its
                tag attribute contains other tag name(e.g: tag = "nightly, smoke, ignore")

Usage of lintest automation framework:
    1.run CMD: "pip install lintest" to install.

    2.copy the run.py to your project
        run.py is an entry to lintest automation framework.
        the structure of your project should be like below:
        ----project_name/     # project name
            ----common/       # common libraries
            ----settings/     # settings will be auto generated by lintest if doesn't exists
            ----tests/        # all the testcases must be under "tests" package
            ----run.py        # the entry of lintest

    3.write TestCases(Frontend or Backend  or MobileTest) under your_project/tests/  - see the testcase demo below

    4.the testcase Class must be extends from UITestCase or APITestCase or SomeTestCase which extends from BaseTestCase

    5.all the TestCases must encapsulate its business logic into a specific method which called run_test()

    6.set the tag for each testcase (tag can be any non-blank string, you can identify it follow your business.
                                     eg:"smoke", "nightly", "ignore" etc.
                                     more details please see setting/testcase_tags.py)

    7.there are 6 ways to execute testcase:
        7.1 run CMD: "python run.py tag_name" (here provide the tag_name, then will execute all the testcase which has
                                    been set tag as tag_name)
         or
        7.2 run CMD: "python run.py TestCaseClassName" (here provide the testcase's class_name: TestCaseClassName(case-insensitive),
                                    then will execute testcase TestCaseClassName)
         or
        7.3 run CMD: "python run.py testcase_list_name" (provide the testcase_list_name which is defined in each package's
                                    __init__.py file)
         or
        7.4 run CMD: "python run.py package_name" (provide the package_name such as: tests.backend  or tests or
                                    tests.backend.b1)
         or
        7.5 run CMD: 'python3 run.py priority=10'
            Run all testcases with priority set and the value is greater than or equal to 10

        7.6 run CMD: "python run.py tag1 tag2 testcase_list_1 testcase_list_2 TestcaseClassName1 package_name_1" (the
                parameters for main.py can be any combination of the following four kinds which separated by a space
                between the parameters:
                    1. tag_name  (e.g: "smoke" or "regression"
                    2. testcase's class_name (e.g: "TestBackendB1_1")
                    3. testcase_list_name  (e.g: "tests_backend_testcases" which in __init__.py)
                    4. package_name (e.g: "tests" or "tests.backend" or "tests.ui")

                all the testcases which fetched by the given parameters will be added into testcase_list, and will do
                data deduplication for testcase_list.

                for each parameter:
                    first try to find it as a tag name, if no tag name found then try to find it as a TestcaseClassName,
                    if no TestcaseClassName found, then try to find it as a TestcaseList, if no TestcaseList found, then
                    try to find it as a package_name

        +++ One More +++
        7.7 run CMD: 'python3 run.py env=DEV threads=10 tests.api'
            means: (Start 10 threads to run the testcases corresponding to tests.api in the DEV environment)


    Tips:
        1. run CMD: "python doctor.py" to check your projects structure.

        2. set testcase_id in run_test(), e.g:

            class Testcase1(APITestCase):
                tag = 'case1, smoke'
                def run_test(self):
                    self.testcase_id = "case1"

        3. there are 2 ways to set the tag for testcase
            1. tag = "smoke, nightly, lin" # set tag as a string(tag names split by ","), follow this way you need
                                           # to make sure all the testcase set the correct tag name( sometimes,
                                           # maybe type error eg. set "nighty" as "nightly")
            2. tag = [Tags.Smoke, Tags.Nightly] # recommend! set tag as a list(tag names defined in settings/testcase_tags.py)
            Note:
                1. if the tag contains "ignore", then this testcase will be only added into ignore_testcase_list(even it contains other tags)
                2. the tag name is Not Case Sensitive, that means tag = "smoke" same with tag = "SMOKE"
                3. if the tag name contains "." (eg. 'lin.wang') then "." will be replace with "_" ('lin_wang')

        4. can set timeout(integer seconds) attribute for each testcase, then it will override the settings.TESTCASE_TIMEOUT
           if not set timeout for testcase, then the testcase's timeout will be default value: settings.TESTCASE_TIMEOUT

        5. if settings.DEBUG_RUN is True then will not generate report.html after each testcase execution done so will
           get a better performance because no "report_lock".
           if run/debug testcase locally its better to set settings.DEBUG_RUN = True.

        6. auto_generate_testcase_list.py can auto generate all the __init__.py files which under tests package. and will
           auto generate all the testcase list which group by tag name.


    ------ API Testcase demo ------:
        from lintest.api_testcase import APITestCase


        class Testcase1(APITestCase):
            tag = 'regression ,smoke'
            priority = 10
            timeout = 20  # seconds

            def run_test(self):
                self.testcase_id = "case1"
                self.sleep(1)
                self.logger.info("Testcase1 executed...")
                self.GlobalData["username"] = "lin"
                self.CaseData["username"] = "lin"
                self.CaseData["phone"] = 110
                self.assert_equals(1, 1)
                self.requests.get(self, "https://www.bing.com")

    ------ Frontend/UI Testcase demo ------:
        from settings.testcase_tags import Tags
        from lintest.frontend_testcase import UITestCase

        class UITestcaseDemo(UITestCase):
            # set tag by way 1, this testcase will be auto added into smoke_testcase_list and nightly_testcase_list
            #tag = "smoke, nightly"

            # set tag by way 2, it's easy to know all the tag names in your project and will not type an error
            # tag name(eg. type "nighty" as "nightly"), recommend this way!
            # this testcase will be auto added into lin_testcase_list & smoke_testcase_list & nightly_testcase_lit
            tag = [Tags.Smoke, Tags.Nightly]

            # here set timeout for testcase, otherwise this testcase's timeout will be: settings.TESTCASE_TIMEOUT
            timeout = 10

            def run_test(self):
                self.testcase_id = "Testcase-1"
                self.logger.debug("load https://www.google.com")
                self.browser.get("https://www.google.com")
        or:

        class UITestcaseDemo(UITestCase):
            #tag = "nightly"  # this testcase will be auto added into nightly_testcase_list
            tag = [Tags.Nightly] # recommend this way

            # here set timeout for testcase, otherwise this testcase's timeout will be : settings.TESTCASE_TIMEOUT
            timeout = 10

            def run_test(self):
                self.logger.debug("load https://www.google.com")
                self.browser.get("https://www.google.com")


        Note: if one testcase has multiple browsers, follow below way, all the browsers will be closed after execution done.

        class FrontendTestcaseDemo(UITestCase):
            tag = [Tags.Ignore]  # this testcase will be auto added into ignore_testcase_list

            # here set timeout for testcase, otherwise this testcase's timeout will be: settings.TESTCASE_TIMEOUT
            timeout = 10

            def run_test(self):
                self.logger.debug("browser1 load https://www.google.com")
                self.browser.get("https://www.google.com")
                self.browser2 = SeleniumHelper("chrome")
                self.logger.debug("browser2 load https://www.github.com")
                Self.browser2.get("https://www.github.com")


    ------ Frontend/UI Testcase (set Token) demo ------:
        from lintest.ui_testcase import UITestCase

        class OrderListWithToken(UITestCase):
            tag = "order, tc9527"

            token = "token-xxx"

            def run_test(self):
                self.browser.get("the-url-which-access-require-a-token")


    ------ Backend/API Testcase Demo ------:
        from settings.testcase_tags import Tags
        from lintest.api_testcase import APITestCase

        class APITestcaseDemo(APITestCase):
            # tag = "lin, smoke"  # this testcase will be auto added into lin_testcase_list & smoke_testcase_list
            tag = [Tags.Lin, Tags.Nightly, Tags.Smoke]  # recommend!

            def run_test(self):
                self.testcase_id = "APITestcase-1"
                self.logger.debug("just for testing")

    ------ Backend/API Testcase  DataSet Demo ------:
    auto generate the testcase list by the .csv & related .py file

    如果需要使用 dataSet(目前只支持CSV格式）,请按照如下方式定义：
        1. 定义一个 csv文件，该文件中必须包含id列
        2. 在csv文件的同级目录下定义一个 和csv文件同名的.py文件，py文件中必须定义一个 run_test(self, param...)方法
        参考如下：
           1. testLogin.csv 文件如下：
               id,channel,username,password
                1,web,"admin1","admin1"
                2,web,admin2,"admin2"
                3,web,user1,user2
                4,app,admin1,admin1
                5,app,admin2,admin2

           2. testLogin.py文件内容如下：
                from common.test_login import login # 引入的具体业务逻辑处理方法

                def run_my_test(self, username, password, channel): # 定义接受的参数（此处接收的参数都必须定义在csv中--列名）
                    self.logger.info("start run_test()...")
                    login(self, username, password, channel)

          -----> 如果正确运行后，框架会根据 testLogin.csv & testLogin.py 自动生成符合框架运行标准的 auto_generated_by_csv.py文件（testLogin_case_list_auto_generated_by_csv.py）
            注意：和__init__.py文件类似，每次执行，都会根据相应的 .csv & .py文件自动生成最新的 auto_generated_by_csv.py文件
          testLogin_case_list_auto_generated_by_csv.py 内容如下：

            from lintest.api_testcase import APITestCase
            from tests.api.csv_dataset import testLoginDataSet


            class testLoginDataSet_1(APITestCase):
                tag = "testLoginDataSet"

                def run_test(self):
                    username = 'admin1'
                    password = 'admin1'

                    testLoginDataSet.run_test(self, username, password)


            class testLoginDataSet_2(APITestCase):
                tag = "testLoginDataSet"

                def run_test(self):
                    username = 'admin2'
                    password = 'admin2'

                    testLoginDataSet.run_test(self, username, password)


            class testLoginDataSet_3(APITestCase):
                tag = "testLoginDataSet"

                def run_test(self):
                    username = 'user1'
                    password = 'user2'

                    testLoginDataSet.run_test(self, username, password)


            class testLoginDataSet_4(APITestCase):
                tag = "testLoginDataSet"

                def run_test(self):
                    username = 'admin1'
                    password = 'admin1'

                    testLoginDataSet.run_test(self, username, password)


            class testLoginDataSet_5(APITestCase):
                tag = "testLoginDataSet"

                def run_test(self):
                    username = 'admin2'
                    password = 'admin2'

                    testLoginDataSet.run_test(self, username, password)


    ------ IOS Testcase Demo ------:
        import os
        from settings.testcase_tags import Tags
        from lintest.frontend_testcase import UITestCase
        from lintest.ios_testcase import IOSTestCase

        class IOSTestCaseDemo(IOSTestCase, UITestCase):
            tag = [Tags.Demo]

            # here set appium_server_ip for testcase, otherwise use default value: settings.appium_server_default_ip
            appium_server_ip = "127.0.0.1"

            # here set appium_server_port for testcase, otherwise use default value: settings.appium_server_default_port
            appium_server_port = "4729"

            ios_desired_capabilities = {
                'app': os.path.abspath('/Users/lin/apps/ikea.app'),
                'platformName': 'iOS',
                'platformVersion': '9.3',
                'deviceName': 'iPhone 6'
            }

            def run_test(self):
                # self.logger is extends from BaseTestCase which was auto generated by framework
                self.logger.debug("start to run ios testcase")

                # self.ios_driver is extends from IOSTestCase which was auto generated by framework
                self.ios_driver.switch_to_alert().accept()

                self.logger.debug("start to load https://ikea.com on PC")
                # self.browser is extends from UITestCase which was auto generated by framework
                self.browser.get("https://ikea.com")

    ------ Android Testcase Demo ------:
        from lintest.android_testcase import AndroidTestCase

        class AndroidTestCaseDemo(AndroidTestCase):
            tag = "android_testcase_demo"

            def run_test(self):
                self.testcase_id = "1"

                user_name = self.android_driver.find_element_by_id("com.ikea2015:id/username")
                user_name.send_keys("user_name_not_exists")
                password = self.android_driver.find_element_by_id("com.ikea2015:id/password")
                password.send_keys("1")

        #Note:
            1. for android testcase, must provide the app path in settings file
            2. all the android testcases must be under package: "tests.android"

    ------ Define Multiple Testcase Classes In One Module ------:
        '''
        lintest support to define multiple Testcase Classes in one module here define 3 testcases(Class Names) in module: testcase_demo.py
        '''
        from settings.testcase_tags import Tags
        from lintest.backend_testcase import APITestCase


        class TestcaseDemo1(APITestCase):
            tag = [Tags.Smoke]

            def run_test(self):
                self.testcase_id = "1"
                self.logger.info("testcase TestcaseDemo1 start ...")


        class TestcaseDemo2(APITestCase):
            tag = [Tags.Smoke]

            def run_test(self):
                self.testcase_id = "2"
                self.logger.info("testcase TestcaseDemo2 start ...")


        class TestcaseDemo3(APITestCase):
            tag = [Tags.Smoke]

            def run_test(self):
                self.testcase_id = "3"
                self.logger.info("testcase TestcaseDemo3 start ...")

@author: Wang Lin,  think_wl@163.com
"""

import importlib
import os
import re
import sys
import json
import traceback
import time
import platform
import datetime
import logging
import threading
import collections
import subprocess
import functools
import lintest

from . import get_project_info

project_info = get_project_info.get_project_info()

try:
    from . import doctor

    doctor.doctor_check()
except BaseException:
    traceback.print_exc()

try:
    import settings
except BaseException:
    raise ("No settings module found ....")

from . import html_report
from . import run_testcase_thread
from . import timeout_thread
from . import testcase_timeout_exception
from . import auto_generate_testcase_list_from_csv
from . import auto_generate_testcase_list
from . import xml_report
from . import doctor
from . import base_testcase
from . import ui_testcase
from . import ios_testcase
from . import get_platform_info
from . import appium_utils
from . import android_testcase
from . import api_testcase
# from .scp_report_to_specified_path import scp_report_to_specified_path
from .base_testcase import BaseTestCase
from .get_adb_devices import get_adb_devices
# from .framework_log import framework_logger
from .xml_report import convert_to_seconds

from urllib.error import URLError

PLATFORM_INFO = get_platform_info.get_platform_info()
DEFAULT_SLEEP_TIME_FOR_MOBILE_TESTCASE = 2
REAL_DEVICE = android_testcase.AndroidTestCase.REAL_DEVICE
VIRTUAL_DEVICE = android_testcase.AndroidTestCase.VIRTUAL_DEVICE


class TestCaseExecutor(threading.Thread):
    # rerun_failed_testcases = []
    # rerun_passed_testcases = []
    failed_testcases = []
    passed_testcases = []
    error_testcases = []
    executing_ios_testcase_flag = False
    testcase_queue = collections.deque()
    report_lock = threading.Lock()
    appium_lock = threading.Lock()
    create_log_lock = threading.Lock()
    db_lock = threading.Lock()
    jenkins_job_name = None
    rerun_testcase_queue = None

    if hasattr(settings, "RERUN_FLAG"):
        if settings.RERUN_FLAG:
            rerun_testcase_queue = collections.deque()

    def __init__(self, testcase_queue, total_testcases_count, output_folder, begin_time):
        threading.Thread.__init__(self)
        self.output_folder = output_folder
        self.begin_time = begin_time
        self.queue = testcase_queue
        self.total_testcases_count = total_testcases_count
        TestCaseExecutor.testcase_queue = self.queue

    def create_logger(self, logger_ref, testcase_full_folder, logger_name):

        with TestCaseExecutor.create_log_lock:
            importlib.reload(logging)

            if not os.path.exists(testcase_full_folder):
                os.makedirs(testcase_full_folder)

            logfile_full_name = testcase_full_folder + os.sep + logger_name

            # Define the log format
            log_format = (
                '[%(asctime)s] %(levelname)-8s %(name)-12s %(message)s')

            handlers = [
                logging.FileHandler(logfile_full_name),
            ]
            if hasattr(settings, "SHOW_LOG_IN_CONSOLE") and settings.SHOW_LOG_IN_CONSOLE is True:
                handlers = [
                    logging.FileHandler(logfile_full_name),
                    logging.StreamHandler(sys.stdout)
                ]

            # Define basic configuration
            logging.basicConfig(
                # Define logging level
                level=logging.DEBUG,
                # Declare the object we created to format the log messages
                format=log_format,
                # Declare handlers
                handlers=handlers
            )

            # Define your own logger name
            logger = logging.getLogger(logger_ref)

            return logger, logfile_full_name

    @staticmethod
    def get_device_name(testcase):
        device_name = testcase.__module__.split(".")[-1]
        if device_name.startswith(VIRTUAL_DEVICE):
            device_name = device_name.split(VIRTUAL_DEVICE)[1]
        elif device_name.startswith(REAL_DEVICE):
            device_name = device_name.split(REAL_DEVICE)[1]

        return device_name

    def run(self):
        while True:
            testcase = None
            if len(TestCaseExecutor.testcase_queue) == 0:
                print("----------------- testcase queue is empty now! -----------------")
                if TestCaseExecutor.rerun_testcase_queue is not None:
                    if len(TestCaseExecutor.rerun_testcase_queue) == 0:
                        print("----------------- rerun testcase queue is empty now! -----------------")
                        break
                    else:
                        testcase = TestCaseExecutor.rerun_testcase_queue.pop()
                        # rerun, set testcase.rerun_tag = 1
                        testcase.rerun_tag = 1
                else:
                    break
            else:
                testcase = TestCaseExecutor.testcase_queue.pop()
                # first run, set testcase.rerun_tag = 0
                testcase.rerun_tag = 0

            testcase.packages = ""
            tc_module_full_path = testcase.__module__

            if tc_module_full_path == "tests":
                testcase.packages = ",tests,"

            if "." in tc_module_full_path:
                package_list = tc_module_full_path.split(".")
                package_name_str_list = ["tests"]
                package_name_str_list_for_db = ""
                p_name_str = "tests"

                for package_name in package_list[1:]:
                    p_name_str += "." + package_name
                    package_name_str_list.append(p_name_str)

                for package_name in package_name_str_list:
                    package_name_str_list_for_db += package_name + ","

                testcase.packages = "," + package_name_str_list_for_db

            tags = getattr(testcase, "tag", "null")

            testcase.tag = ""

            if isinstance(tags, str):
                if tags == "null":
                    testcase.tag = tags
                else:
                    if "," in tags:
                        tags = tags.strip(",")
                        tags = tags.replace(" ", "")
                    testcase.tag = "," + tags.lower() + ","

            if isinstance(tags, list):
                if len(tags) == 0:
                    testcase.tag = "null"
                else:
                    for tag in tags:
                        testcase.tag += tag.lower().strip() + ","
                    testcase.tag = "," + testcase.tag

            if issubclass(testcase, ios_testcase.IOSTestCase):
                if TestCaseExecutor.executing_ios_testcase_flag is True:
                    TestCaseExecutor.testcase_queue.appendleft(testcase)
                    print(
                        "------ when TestcaseExecutor: %s running ----- already one ios testcase executing" % self)
                    time.sleep(DEFAULT_SLEEP_TIME_FOR_MOBILE_TESTCASE)
                    continue
                else:
                    TestCaseExecutor.executing_ios_testcase_flag = True

            if issubclass(testcase, android_testcase.AndroidTestCase):
                device_name = TestCaseExecutor.get_device_name(testcase)

                if getattr(TestCaseExecutor, device_name) is True:
                    TestCaseExecutor.testcase_queue.appendleft(testcase)
                    # TestCaseExecutor.testcase_queue.append(testcase)
                    print(
                        "------ already one android testcase executing on device: %s" % device_name)
                    time.sleep(DEFAULT_SLEEP_TIME_FOR_MOBILE_TESTCASE)
                    continue
                else:
                    setattr(TestCaseExecutor, device_name, True)

            try:
                executing_testcase = testcase()
                print("----- Testcase: %s is running ---------" % executing_testcase)
                executing_testcase.timeout_flag = False
                executing_testcase.run_flag = False
                executing_testcase.done_flag = False
                executing_testcase.rerun_tag = testcase.rerun_tag

                if not hasattr(executing_testcase, "screenshot_list_for_db"):
                    executing_testcase.screenshot_path_list_for_db = []

                executing_testcase.ExecutionStatusSetByFramework = None

                if isinstance(executing_testcase, ui_testcase.UITestCase) or \
                        isinstance(executing_testcase, ios_testcase.IOSTestCase) or \
                        isinstance(executing_testcase, android_testcase.AndroidTestCase):
                    if executing_testcase.rerun_tag == 0:
                        executing_testcase.screenshot_name = "%s_screenshot.png" % testcase.__name__
                    else:
                        executing_testcase.screenshot_name = "%s_screenshot.rerun.png" % testcase.__name__

                executing_testcase.testcase_start_time = datetime.datetime.now()
                testcase_full_folder = self.output_folder + os.sep + testcase.__module__.replace(".", "_") + \
                                       os.sep + testcase.__name__

                if not os.path.exists(testcase_full_folder):
                    os.makedirs(testcase_full_folder)

                executing_testcase.full_tc_folder = testcase_full_folder

                if executing_testcase.rerun_tag == 0:
                    logger, logfile_full_name = self.create_logger(testcase.__name__,
                                                                   executing_testcase.full_tc_folder,
                                                                   "test.log")
                    executing_testcase.logger = logger

                    executing_testcase.logfile_full_name = logfile_full_name
                    executing_testcase.log_file_path = logfile_full_name.replace(project_info.output_folder, ".")

                else:
                    logger_rerun, logfile_full_name_rerun = self.create_logger(testcase.__name__,
                                                                               executing_testcase.full_tc_folder,
                                                                               "test.rerun.log")
                    executing_testcase.logger = logger_rerun
                    executing_testcase.logfile_full_name = logfile_full_name_rerun
                    executing_testcase.log_file_path = logfile_full_name_rerun.replace(project_info.output_folder, ".")

                print(executing_testcase.logger)
                executing_testcase.logger.info(
                    "====================== TestCase: {TestCaseName} Execution Log ======================".format(
                        TestCaseName=testcase.__name__) + os.linesep)

                executing_testcase.logger.info("---------------- Platform: %s" % PLATFORM_INFO + os.linesep)
                executing_testcase.logger.info("---------------- Test Framework: lintest==%s" % lintest.__version__ + os.linesep)
                executing_testcase.logger.info(
                    "---------------- Environment: %s " % settings.ENVIRONMENT.upper() + os.linesep)

                # if IOS testcase run then should start appium first
                if issubclass(testcase, ios_testcase.IOSTestCase) or issubclass(testcase,
                                                                                android_testcase.AndroidTestCase):
                    from appium import webdriver as mobiledriver

                    xcode_version = get_platform_info.get_xcode_version()
                    print("xcode_version: %s" % xcode_version)

                    logfile_full_name_for_appium = executing_testcase.logfile_full_name.replace(".log", ".appium_log")

                    if hasattr(testcase, "appium_server_ip"):
                        appium_server_ip = testcase.appium_server_ip
                    else:
                        try:
                            appium_server_ip = settings.appium_server_default_ip
                        except AttributeError:
                            appium_server_ip = appium_utils.APPIUM_SERVER_DEFAULT_IP
                        testcase.appium_server_ip = appium_server_ip

                    if hasattr(testcase, "appium_server_port"):
                        appium_server_port = testcase.appium_server_port
                    else:
                        try:
                            appium_server_port = settings.appium_server_default_port
                        except AttributeError:
                            appium_server_port = appium_utils.APPIUM_SERVER_DEFAULT_PORT
                        testcase.appium_server_port = appium_server_port

                    testcase.appium_server_ip = appium_server_ip
                    testcase.appium_server_port = appium_server_port

                    # if testcase.rerun_tag == 1:
                    #     testcase.appium_server_port = str(int(testcase.appium_server_port) + 10000)

                    # because after start appium, need get the node process_id, so here add appium_lock
                    with TestCaseExecutor.appium_lock:
                        appium_utils.start_appium(testcase,
                                                  appium_log_path=logfile_full_name_for_appium,
                                                  appium_server_ip=testcase.appium_server_ip,
                                                  appium_server_port=testcase.appium_server_port)

                if issubclass(testcase, ios_testcase.IOSTestCase):
                    if hasattr(settings, "ios_desired_capabilities_in_settings_first"):
                        if settings.ios_desired_capabilities_in_settings_first is True:
                            if hasattr(settings, "ios_desired_capabilities"):
                                executing_testcase.logger.info(settings.ios_desired_capabilities)
                                testcase.ios_desired_capabilities = settings.ios_desired_capabilities
                            else:
                                if hasattr(testcase, "ios_desired_capabilities"):
                                    executing_testcase.logger.info(testcase.ios_desired_capabilities)
                                else:
                                    raise Exception(
                                        "There are no ios_desired_capabilities found in both testcase & settings, Please make sure provide ios_desired_capabilities in settings or testcase")
                        else:
                            if hasattr(testcase, "ios_desired_capabilities"):
                                executing_testcase.logger.info(testcase.ios_desired_capabilities)
                            else:
                                if hasattr(settings, "ios_desired_capabilities"):
                                    # executing_testcase.logger.info(settings.ios_desired_capabilities)
                                    testcase.ios_desired_capabilities = settings.ios_desired_capabilities
                                else:
                                    raise Exception(
                                        "There are no ios_desired_capabilities found in both testcase & settings, Please make sure provide ios_desired_capabilities in settings or testcase")
                    else:
                        if hasattr(testcase, "ios_desired_capabilities"):
                            executing_testcase.logger.info(testcase.ios_desired_capabilities)
                        else:
                            if hasattr(settings, "ios_desired_capabilities"):
                                executing_testcase.logger.info(settings.ios_desired_capabilities)
                                testcase.ios_desired_capabilities = settings.ios_desired_capabilities
                            else:
                                raise Exception(
                                    "There are no ios_desired_capabilities found in both testcase & settings, Please make sure provide ios_desired_capabilities in settings or testcase")

                    if int(xcode_version.split(".")[0]) >= 8:
                        if "automationName" not in testcase.ios_desired_capabilities.keys():
                            testcase.ios_desired_capabilities["automationName"] = "XCUITest"

                    if "platformName" not in testcase.ios_desired_capabilities.keys():
                        testcase.ios_desired_capabilities["platformName"] = "ios"

                    if not "platformVersion" in testcase.ios_desired_capabilities.keys():
                        testcase.ios_desired_capabilities[
                            "platformVersion"] = get_platform_info.get_ios_platform_version()

                    if "app" in testcase.ios_desired_capabilities.keys():
                        app_path = testcase.ios_desired_capabilities["app"]
                        if app_path.startswith("/"):
                            # if the app_path starts_with "/", then it means user set the absolute path, just pass it.
                            pass
                        else:
                            if app_path.startswith("app/"):
                                testcase.ios_desired_capabilities["app"] = project_info.project_path + "/" + app_path
                            else:
                                # use the package (e.g: app: "com.ikea")
                                pass

                    executing_testcase.ios_driver = mobiledriver.Remote(
                        'http://%s:%s/wd/hub' % (testcase.appium_server_ip, testcase.appium_server_port),
                        desired_capabilities=testcase.ios_desired_capabilities)
                    executing_testcase.logger.info(
                        '---------- IOS Driver Info: %s' % executing_testcase.ios_driver.__dict__ + os.linesep)

                    # log_memory_usage("after create ios driver", executing_testcase)

                    if hasattr(settings, "ios_implicitly_wait"):
                        if type(settings.ios_implicitly_wait) == int or type(settings.ios_implicitly_wait) == float:
                            executing_testcase.ios_driver.implicitly_wait(getattr(settings, "ios_implicitly_wait"))
                            executing_testcase.logger.info("set ios_implicitly_wait:%s" % settings.ios_implicitly_wait)
                        else:
                            # if the type of settings.ios_implicitly_wait is not correct, here set a default value: 60
                            executing_testcase.ios_driver.implicitly_wait(60)
                            print(
                                "the type of settings.ios_implicitly_wait:%s is not 'int' or 'float', here set a default value: 60" % settings.ios_implicitly_wait)
                            executing_testcase.logger.warning(
                                "the type of settings.ios_implicitly_wait:%s is not 'int' or 'float', here set a default value: 60" % settings.ios_implicitly_wait)
                    else:
                        # if there are no ios_implicitly_wait found in settings, then set a default value: 60
                        executing_testcase.ios_driver.implicitly_wait(60)
                        print(
                            "there are no ios_implicitly_wait found in settings, then set a default value: 60")
                        executing_testcase.logger.warning(
                            "there are no ios_implicitly_wait found in settings, then set a default value: 60")

                if issubclass(testcase, android_testcase.AndroidTestCase):
                    if hasattr(testcase, "logger"):
                        if hasattr(testcase, "android_desired_capabilities"):
                            executing_testcase.logger.info(testcase.android_desired_capabilities)
                        else:
                            executing_testcase.logger.info(settings.android_desired_capabilities)
                            testcase.android_desired_capabilities = settings.android_desired_capabilities

                    executing_testcase.android_driver = mobiledriver.Remote(
                        'http://%s:%s/wd/hub' % (testcase.appium_server_ip, testcase.appium_server_port),
                        desired_capabilities=testcase.android_desired_capabilities)

                    if hasattr(settings, "android_implicitly_wait"):
                        if type(settings.android_implicitly_wait) == int or type(
                                settings.android_implicitly_wait) == float:
                            executing_testcase.android_driver.implicitly_wait(
                                getattr(settings, "android_implicitly_wait"))
                            executing_testcase.logger.info(
                                "set android_implicitly_wait:%s" % settings.android_implicitly_wait)
                        else:
                            # if the type of settings.android_implicitly_wait is not correct, here set a default value: 60
                            print(
                                "the type of settings.android_implicitly_wait:%s is not 'int' or 'float', here set a default value: 60" % settings.android_implicitly_wait)
                            executing_testcase.logger.warning(
                                "the type of settings.android_implicitly_wait:%s is not 'int' or 'float', here set a default value: 60" % settings.android_implicitly_wait)
                    else:
                        # if there are no android_implicitly_wait found in settings, then set a default value: 60
                        executing_testcase.android_driver.implicitly_wait(60)
                        print(
                            "there are no android_implicitly_wait found in settings, then set a default value: 60")
                        executing_testcase.logger.warning(
                            "there are no android_implicitly_wait found in settings, then set a default value: 60")

                    executing_testcase.logger.info(
                        '---------- Android Driver Info: %s' % executing_testcase.android_driver.__dict__ + os.linesep)

                    # log_memory_usage("after create android driver", executing_testcase)

                # Backend testcases will not need to create browser & close browser
                if issubclass(testcase, ui_testcase.UITestCase):
                    try:
                        executing_testcase.create_browser_driver()
                        testcase.browser = executing_testcase.browser
                        testcase.browserName = executing_testcase.browser.name
                        executing_testcase.logger.info(
                            "---------------- Browser: %s - Version:%s " % (
                                testcase.browserName.upper(),
                                testcase.browser.capabilities['browserVersion']) + os.linesep)
                        # log_memory_usage("after create browser driver", executing_testcase)
                    except URLError:
                        traceback.print_exc()
                        raise BaseException("open browser error")

                # todo: DEBUG run 模式 真需要吗？！
                if not settings.DEBUG_RUN:
                    # if settings.DEBUG_RUN is False, sometimes maybe the
                    # browser will in "No response" Status, so need to set timeout for each testcase.
                    # here start 2 daemon thread to implement the testcase's timeout function

                    testcase.timeout_flag = False
                    testcase.run_flag = False
                    testcase.done_flag = False

                    testcase_timeout_thread = timeout_thread.TimeoutThread(executing_testcase)
                    testcase_timeout_thread.setDaemon(True)
                    testcase_timeout_thread.start()

                    if hasattr(executing_testcase, "exception_info"):
                        # before testcase start to run, set its exception_info as None
                        executing_testcase.exception_info = None

                    while executing_testcase.timeout_flag is False and executing_testcase.done_flag is False:
                        if executing_testcase.run_flag is False:
                            executing_testcase.run_flag = True

                            testcase_run_testcase_thread = run_testcase_thread.StartTestcaseThread(executing_testcase)
                            testcase_run_testcase_thread.setDaemon(True)
                            testcase_run_testcase_thread.start()

                            if hasattr(executing_testcase, "exception_info"):
                                if executing_testcase.exception_info is not None:
                                    raise Exception(executing_testcase.exception_info)

                        if hasattr(executing_testcase, "exception_info"):
                            if executing_testcase.exception_info is not None:
                                raise Exception(executing_testcase.exception_info)

                    if executing_testcase.timeout_flag is True:
                        raise testcase_timeout_exception.TestCaseTimeoutException("TimeoutException")
                else:
                    # 执行 setup() 之前 先 log TestEngineCaseInput(如果是 test-engine 发起的)
                    if BaseTestCase.TestEngineCaseInput:
                        executing_testcase.logger.info("------ TestEngineCaseInput:")
                        executing_testcase.logger.info(executing_testcase.TestEngineCaseInput)

                    # 先执行 setup()， 再执行 run_test()
                    try:
                        if hasattr(executing_testcase, "setup"):
                            executing_testcase.logger.info("------ execute setup() Start")
                            executing_testcase.setup()
                            executing_testcase.logger.info("------ execute setup() End")
                    except BaseException:
                        pass

                    executing_testcase.logger.info("------ run_test() Start")

                    # For DEBUG_RUN mode,will not set timeout for each testcase
                    executing_testcase.run_test()

            except BaseException:
                executing_testcase.ExecutionStatusSetByFramework = "failed"
                # log_memory_usage("catch exception", executing_testcase)
                traceback.print_exc()
                # print(testcase.__dict__)
                traceback_info = traceback.format_exc()

                if issubclass(testcase, ui_testcase.UITestCase):

                    try:
                        # save the current URL and all the browser's related information when got error.
                        executing_testcase.logger.error(
                            "---------------- Error URL & Browser Info Start --------------" +
                            testcase.browser.current_url + os.linesep)
                        executing_testcase.logger.error(testcase.browser.__dict__)
                        executing_testcase.logger.error(
                            "---------------- Error URL & Browser Info End --------------" + os.linesep)
                    except:
                        traceback.print_exc()

                executing_testcase.logger.error(traceback_info)

                if executing_testcase.timeout_flag is True:
                    exception_info = "TestCaseTimeoutException"
                    traceback_info_str = "TestcaseTimeout"
                else:
                    traceback_info_reversed = []

                    for line in traceback_info.strip().splitlines():
                        traceback_info_reversed.insert(0, line)

                    for traceback_info_str in traceback_info_reversed:
                        exception_info = traceback_info_str.split(":")[0]

                        if exception_info.__contains__("Exception") or exception_info.__contains__("Error"):
                            # remove the html tag in execption_info
                            exception_info = re.sub(r'</?\w+[^>]*>', '', exception_info)
                            break

                executing_testcase.exception_info = exception_info
                executing_testcase.exception_info_for_xml_report = re.sub(r'</?\w+[^>]*>', '', traceback_info_str)
                executing_testcase.error_message_for_db = executing_testcase.exception_info_for_xml_report
                # testcase.exception_info_for_xml_report = traceback_info_str
                executing_testcase.traceback_info = traceback_info

                TestCaseExecutor.failed_testcases.append(executing_testcase)

                if TestCaseExecutor.rerun_testcase_queue is not None:
                    if executing_testcase.rerun_tag == 0:
                        TestCaseExecutor.rerun_testcase_queue.append(testcase)
                        print("TestCase: %s run got failed, Re-run it later..." % testcase.__name__)

                with open(executing_testcase.logfile_full_name, "r") as case_log_file:
                    log_file_content = case_log_file.read()
                    print(log_file_content)
                    executing_testcase.log_file_content = log_file_content
                    print(executing_testcase.log_file_content)

                try:
                    # 只有通过test-engine 触发的end2end testcase 才需要由框架统一调用 test-engine callback-api
                    if BaseTestCase.TestEngineCaseInput:
                        headers = {
                            "Content-Type": "application/json",
                            "Host": "%s" % settings.TestEngineCallBackHost
                        }

                        for key in executing_testcase.TestEngineCaseOutput:
                            if type(executing_testcase.TestEngineCaseOutput[key]) is str or\
                                    type(executing_testcase.TestEngineCaseOutput[key]) is int:
                                pass
                            else:
                                raise Exception("The type of %s can only be str or int " % executing_testcase.TestEngineCaseOutput[key])

                        log_path = project_info.project_name + executing_testcase.logfile_full_name.split(
                            project_info.project_name + os.sep + "output")[-1]
                        print(log_path)

                        callback_post_data = {
                            "chainDataId": executing_testcase.TestEngineCaseInput["execution_id"],
                            "currentCaseIndex": executing_testcase.TestEngineCaseInput["case_index"],
                            "respBody": executing_testcase.TestEngineCaseOutput,
                            "reqBody": executing_testcase.TestEngineCaseInput,
                            # "caseExecutionLog": log_path,
                            "caseExecutionLog": executing_testcase.log_file_content,
                            "testExecutionStatus": "FAILED"
                        }


                        res = executing_testcase.requests.post(
                            executing_testcase,
                            "%s/case/exec/complete" % settings.TestEngineCallBackAPI,
                            data=json.dumps(callback_post_data),
                            headers=headers)

                        # if res.status_code == 200:
                        #     log_file_content = ""
                        #     with open(executing_testcase.logfile_full_name, "r") as case_log_file:
                        #         # log_file_content = case_log_file.read()
                        #         log_path = project_info.project_name + executing_testcase.logfile_full_name.split(
                        #             project_info.project_name + os.sep + "output")[-1]
                        #         print(log_path)
                        #         # todo: remove
                        #         # change to save the log_path instead of log content
                        #         log_file_content = log_path
                        #
                        #     executing_testcase.requests.post("%s/chainData/uploadLog?chainDataId=%s&currentCaseIndex=%s" %(
                        #         settings.TestEngineCallBackAPI,
                        #         executing_testcase.TestEngineCaseInput["execution_id"], executing_testcase.TestEngineCaseInput["case_index"]),
                        #                                 data=log_file_content.encode(),
                        #                                 headers=headers)

                except BaseException:
                    traceback.print_exc()
                    executing_testcase.logger.error(traceback.format_exc())
                    pass
            else:
                # TestCaseExecutor.passed_testcases.append(testcase)
                TestCaseExecutor.passed_testcases.append(executing_testcase)

                for index, failed_tc in enumerate(TestCaseExecutor.failed_testcases):
                    if type(executing_testcase) == type(failed_tc):
                        TestCaseExecutor.failed_testcases.pop(index)
                        break

                try:
                    # 只有通过test-engine 触发的end2end testcase 才需要由框架统一调用 test-engine callback-api
                    if BaseTestCase.TestEngineCaseInput:
                        headers = {
                            "Content-Type": "application/json",
                            "Host": "%s" % settings.TestEngineCallBackHost
                        }

                        print(executing_testcase.logger)
                        print(executing_testcase.logger.__dict__)
                        with open(executing_testcase.logfile_full_name, "r") as case_log_file:
                            log_file_content = case_log_file.read()
                            print(log_file_content)
                            executing_testcase.log_file_content = log_file_content
                            print(executing_testcase.log_file_content)

                        for key in executing_testcase.TestEngineCaseOutput:
                            if type(executing_testcase.TestEngineCaseOutput[key]) is str or\
                                    type(executing_testcase.TestEngineCaseOutput[key]) is int:
                                pass
                            else:
                                raise Exception("The type of %s can only be str or int " % executing_testcase.TestEngineCaseOutput[key])

                        log_path = project_info.project_name + executing_testcase.logfile_full_name.split(
                            project_info.project_name + os.sep + "output")[-1]
                        print("===will callback log_path")
                        print(log_path)

                        callback_post_data = {
                            "chainDataId": executing_testcase.TestEngineCaseInput["execution_id"],
                            "currentCaseIndex": executing_testcase.TestEngineCaseInput["case_index"],
                            "respBody": executing_testcase.TestEngineCaseOutput,
                            "reqBody": executing_testcase.TestEngineCaseInput,
                            # "caseExecutionLog": log_path,
                            "caseExecutionLog": executing_testcase.log_file_content,
                            "testExecutionStatus": "PASSED"
                        }

                        res = executing_testcase.requests.post(
                            executing_testcase,
                            "%s/case/exec/complete" % settings.TestEngineCallBackAPI,
                            data=json.dumps(callback_post_data),
                            headers=headers)

                        # if res.status_code == 200:
                        #     log_file_content = ""
                        #     with open(executing_testcase.logfile_full_name, "r") as case_log_file:
                        #         # log_file_content = case_log_file.read()
                        #         log_path = project_info.project_name + executing_testcase.logfile_full_name.split(
                        #             project_info.project_name + os.sep + "output")[-1]
                        #         print(log_path)
                        #
                        #         # change to save the log_path instead of log content
                        #         log_file_content = log_path
                        #
                        #     executing_testcase.requests.post(
                        #         "%s/chainData/uploadLog?chainDataId=%s&currentCaseIndex=%s" % (
                        #             settings.TestEngineCallBackAPI,
                        #             executing_testcase.TestEngineCaseInput["execution_id"],
                        #             executing_testcase.TestEngineCaseInput["case_index"]),
                        #         data=log_file_content.encode(),
                        #         headers=headers)
                except BaseException:
                    traceback.print_exc()
                    executing_testcase.logger.error("\n\n")
                    executing_testcase.logger.error("******************************** Invoke TestEngine Callback ******")
                    executing_testcase.logger.error("********************************  Error  *************************")
                    executing_testcase.logger.error(traceback.format_exc())
                    pass
            finally:
                try:
                    print(
                        "TestCase: %s ------ Done (Passed:%s | Failed:%s | Remaining:%s | Total:%s)" % (
                            testcase.__name__,
                            len(TestCaseExecutor.passed_testcases),
                            len(TestCaseExecutor.failed_testcases),
                            len(TestCaseExecutor.testcase_queue),
                            self.total_testcases_count))

                except BaseException:
                    pass

                try:
                    if hasattr(executing_testcase, "teardown"):
                        executing_testcase.logger.info("------ execute teardown() Start")
                        executing_testcase.teardown()
                        executing_testcase.logger.info("------ execute teardown() End")
                except BaseException:
                    pass

                try:
                    if hasattr(executing_testcase, "testcase_id"):
                        testcase.testcase_id = executing_testcase.testcase_id
                    elif hasattr(executing_testcase, "test_case_id"):
                        executing_testcase.logger.warning(
                            "******** Warning: 'test_case_id' has been deprecated, please change to 'testcase_id' ********")
                        testcase.testcase_id = executing_testcase.test_case_id
                    else:
                        testcase.testcase_id = "None"

                    executing_testcase.logger.handlers = []
                    testcase_end_time = datetime.datetime.now()
                    testcase_execution_time = testcase_end_time - executing_testcase.testcase_start_time

                    executing_testcase.execution_time = convert_to_seconds(testcase_execution_time)

                    # backend testcase will not have to create browser & close browser
                    if issubclass(testcase, ui_testcase.UITestCase) or \
                            issubclass(testcase, ios_testcase.IOSTestCase) or \
                            issubclass(testcase, android_testcase.AndroidTestCase):
                        from selenium.webdriver.remote.webdriver import WebDriver

                        webdriver_list = []

                        # log_memory_usage("start to close android/ios/browser driver", executing_testcase)
                        # sometimes one testcase will open multiple browsers(eg. teacher & students), here will fetch all the
                        # testcase's browsers and save screenshot for all browsers then close them.
                        for key, val in executing_testcase.__dict__.items():
                            if isinstance(val, WebDriver):
                                webdriver_list.append(key)

                        close_browser_after_done_flag = True;
                        if hasattr(settings, "CLOSE_BROWSER"):
                            close_browser_after_done_flag = settings.CLOSE_BROWSER

                        for browser in webdriver_list:
                            webdriver = getattr(executing_testcase, browser)

                            try:
                                if executing_testcase.ExecutionStatusSetByFramework == "failed":
                                    webdriver.save_screenshot(
                                        testcase_full_folder + os.sep + browser + "_" +
                                        executing_testcase.screenshot_name)
                                    executing_testcase.screenshot_path_list_for_db.append(
                                        testcase_full_folder + os.sep + browser + "_" +
                                        executing_testcase.screenshot_name)
                                elif hasattr(settings, "SaveScreenshotForPassedTestCaseFlag"):
                                    if settings.SaveScreenshotForPassedTestCaseFlag is True:
                                        webdriver.save_screenshot(
                                            testcase_full_folder + os.sep + browser + "_" +
                                            executing_testcase.screenshot_name)
                                        executing_testcase.screenshot_path_list_for_db.append(
                                            testcase_full_folder + os.sep + browser + "_" +
                                            executing_testcase.screenshot_name)

                                if executing_testcase.rerun_tag == 0:
                                    executing_testcase.screenshot = testcase_full_folder.replace(
                                        project_info.output_folder, ".") \
                                                                    + os.sep + browser + "_" + \
                                                                    executing_testcase.screenshot_name
                                elif executing_testcase.rerun_tag == 1:
                                    executing_testcase.rerun_screenshot = testcase_full_folder.replace(
                                        project_info.output_folder,
                                        ".") + os.sep + "_" + executing_testcase.screenshot_name
                                # if executing_testcase.ExecutionStatusSetByFramework == "failed":
                                #     webdriver.save_screenshot(
                                #         testcase_full_folder + os.sep + webdriver.name + "_" + browser + "_" +
                                #         executing_testcase.screenshot_name)
                                #     executing_testcase.screenshot_path_list_for_db.append(
                                #         testcase_full_folder + os.sep + webdriver.name + "_" + browser + "_" +
                                #         executing_testcase.screenshot_name)
                                # elif hasattr(settings, "SaveScreenshotForPassedTestCaseFlag"):
                                #     if settings.SaveScreenshotForPassedTestCaseFlag is True:
                                #         webdriver.save_screenshot(
                                #             testcase_full_folder + os.sep + webdriver.name + "_" + browser + "_" +
                                #             executing_testcase.screenshot_name)
                                #         executing_testcase.screenshot_path_list_for_db.append(
                                #             testcase_full_folder + os.sep + webdriver.name + "_" + browser + "_" +
                                #             executing_testcase.screenshot_name)
                                #
                                # if executing_testcase.rerun_tag == 0:
                                #     executing_testcase.screenshot = testcase_full_folder.replace(
                                #         project_info.output_folder, ".") \
                                #                                     + os.sep + webdriver.name + "_" + browser + "_" + \
                                #                                     executing_testcase.screenshot_name
                                # elif executing_testcase.rerun_tag == 1:
                                #     executing_testcase.rerun_screenshot = testcase_full_folder.replace(
                                #         project_info.output_folder,
                                #         ".") + os.sep + webdriver.name + "_" + browser + "_" + \
                                #                                           executing_testcase.screenshot_name
                            except BaseException:
                                traceback.print_exc()
                            finally:
                                try:
                                    is_appium_installed = True
                                    try:
                                        import appium
                                    except ImportError:
                                        is_appium_installed = False

                                    if is_appium_installed is True:
                                        # if IOS testcase or Andriod Testcase
                                        if (issubclass(executing_testcase.__class__, ios_testcase.IOSTestCase) or
                                            issubclass(executing_testcase.__class__, android_testcase.AndroidTestCase)) \
                                                and isinstance(webdriver, appium.webdriver.webdriver.WebDriver):
                                            try:
                                                webdriver.close_app()
                                            except BaseException:
                                                traceback.print_exc()
                                            try:
                                                if hasattr(testcase, "node_process_pid"):
                                                    if platform.system() == "Windows":
                                                        os.system("taskkill /f /im %s" % testcase.node_process_pid)
                                                    else:
                                                        os.system("kill -9 %s" % testcase.node_process_pid)
                                                        # todo: delattr(testcase, "node_process_pid") ? then no need blew one line code?
                                                    testcase.process_id_testcase_dict.pop(testcase.node_process_pid)
                                            except BaseException:
                                                traceback.print_exc()
                                        elif isinstance(webdriver, WebDriver):
                                            if close_browser_after_done_flag:
                                                webdriver.delete_all_cookies()
                                                webdriver.quit()
                                    elif isinstance(webdriver, WebDriver):
                                        if close_browser_after_done_flag:
                                            webdriver.delete_all_cookies()
                                            webdriver.quit()
                                except BaseException:
                                    traceback.print_exc()

                        if issubclass(testcase, ios_testcase.IOSTestCase):
                            # set executing_ios_testcase_flag as False after this ios testcase execution done
                            TestCaseExecutor.executing_ios_testcase_flag = False  # todo
                            kill_process_by_name("Simulator")

                        if issubclass(testcase, android_testcase.AndroidTestCase):
                            setattr(TestCaseExecutor, device_name, False)
                except BaseException:
                    traceback.print_exc()
                finally:
                    try:
                        if issubclass(testcase, ios_testcase.IOSTestCase):
                            # set executing_ios_testcase_flag as False after this ios testcase execution done
                            TestCaseExecutor.executing_ios_testcase_flag = False

                        if issubclass(testcase, android_testcase.AndroidTestCase):
                            setattr(TestCaseExecutor, device_name, False)

                        with open(executing_testcase.logfile_full_name, "r", encoding="utf-8", errors="ignore") as logfile_full_name:
                            execution_log = logfile_full_name.read()
                            executing_testcase.execution_log = execution_log
                    finally:
                        if os.sep + "jenkins" + os.sep + "workspace" + os.sep in project_info.project_path:
                            TestCaseExecutor.jenkins_job_name = \
                                project_info.project_path.split(os.sep + "jenkins" + os.sep + "workspace" + os.sep)[
                                    1].split(os.sep)[0]

                        # #log_memory_usage("start to save log into DB", executing_testcase)

                # #log_memory_usage("start to generate html report", executing_testcase)
                # # if settings.DEBUG_RUN is True then will not generate report.html after each testcase execution done
                # # and will get a better performance because no "report_lock".
                # if settings.DEBUG_RUN is False:
                #     # if settings.DEBUG_RUN is False and
                #     #   1. testcase got passed   or
                #     #   2. testcase got failed but rerun_tag == 1 and settings.RERUN_FLAG is True   or
                #     #   3. testcase got failed and settings.RERUN_FLAG is False
                #     # then: generate the html report after each testcase execution done!
                #     if testcase in TestCaseExecutor.passed_testcases or \
                #             (settings.RERUN_FLAG is True and
                #              testcase in TestCaseExecutor.failed_testcases and testcase.rerun_tag == 1) \
                #             or \
                #             (settings.RERUN_FLAG is False and testcase in TestCaseExecutor.failed_testcases):
                #         pass
                #
                #         # with TestCaseExecutor.report_lock:
                #         #     try:
                #         #         # print(self.begin_time)
                #         #         html_report.Reporter(self.output_folder,
                #         #                              TestCaseExecutor.passed_testcases,
                #         #                              TestCaseExecutor.failed_testcases,
                #         #                              TestCaseExecutor.error_testcases,
                #         #                              self.begin_time,
                #         #                              PLATFORM_INFO,
                #         #                              True)
                #         #     except BaseException:
                #         #         traceback.print_exc()
                #         #         print(traceback.format_exc())
                #
                # #log_memory_usage("after generate the html report", executing_testcase)


def kill_process_by_name(process_name):
    for i in range(5):
        try:
            if platform.system() == "Windows":
                subprocess.check_output("taskkill /f /im %s" % process_name, shell=True, stderr=subprocess.STDOUT)
            else:
                subprocess.check_output("killall '%s'" % process_name, shell=True, stderr=subprocess.STDOUT)

            print("kill process: %s" % process_name)

        except BaseException as e:
            pass


def clean_appium_env():
    if platform.system() == "Windows":
        kill_process_by_name("node.exe")
    else:
        kill_process_by_name("node")
        kill_process_by_name("instruments")
        kill_process_by_name("Simulator")


def close_browsers_and_webdrivers():
    if platform.system() == "Windows":
        kill_process_by_name("chrome.exe")
        kill_process_by_name("chromedriver.exe")
        kill_process_by_name("firefox.exe")
        kill_process_by_name("iexplore.exe")
        kill_process_by_name("IEDriverServer.exe")
        kill_process_by_name("node.exe")
    else:
        # kill_process_by_name("Google Chrome")
        kill_process_by_name("chromedriver")
        kill_process_by_name("node")
        kill_process_by_name("instruments")
        kill_process_by_name("Simulator")


def usage():
    print(os.linesep + "-------------------------------- lintest Automation Framework Usage Start -----------------------------")
    print(__doc__)
    print("-------------------------------- lintest Automation Framework Usage End --------------------------------" + os.linesep)


def update_config(args):
    """
    update_config.py is used to update the settings/__init__.py

    Note:
        the argv must be like "key=val key2=val2", each argv split by white-space and the key is case-insensitive

    @author: Wang Lin
    """

    import sys
    import os

    # from lintest.get_project_info import get_project_info
    from .get_project_info import get_project_info

    # os.environ["LIN_TEST_PROJECT_PATH"] = os.path.dirname(os.path.abspath(__file__))
    # sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))

    project_info = get_project_info()

    # todo: self.project_path = project_info.project_path
    settings_file_path = project_info.project_path + os.sep + "settings" + os.sep + "__init__.py"

    # argv_list = sys.argv[1:]
    argv_list = []

    for item in args.split("&"):
        item = item.lstrip().rstrip()
        if item.find("=") != -1:
            argv_list.append(item)

    if len(argv_list) > 0:
        for argv in argv_list:
            if "=" not in argv:
                print("the argv must like: key=val")
                # raise BaseException("the argv must like: key=val")

        argv_dict = {}
        updated_argv = []

        for argv in argv_list:
            if (argv.find("=") != -1):
                argv_dict[argv.split("=")[0].lstrip().rstrip()] = argv.split("=")[1].lstrip().rstrip()

        with open(settings_file_path, "r") as settings_file:
            contents = settings_file.readlines()
            i = 0

            while i < len(contents):
                line = contents[i]
                line_remove_whitespace = line.replace(" ", "")

                for argv in argv_dict.keys():
                    if line_remove_whitespace.split("=")[0].lower().lstrip().rstrip() == argv.lower().lstrip().rstrip():
                        contents[i] = "%s = %s \n" % (
                            line_remove_whitespace.split("=")[0].lstrip().rstrip(), argv_dict[argv].lstrip().rstrip())
                        print("UPDATE in settings" + os.sep + "__init__.py: %s " % contents[i])
                        updated_argv.append(argv)
                        break
                i += 1

            with open(settings_file_path, "w") as settings_new_file:
                settings_new_file.writelines(contents)

        if len(updated_argv) < len(argv_list):
            argv_list_not_found = []

            for argv in argv_dict.keys():
                if argv not in updated_argv:
                    argv_list_not_found.append(argv)

            print("***************** WARNING: below argv not found in settings/__init__.py *****************")
            print(argv_list_not_found)

        # with open(settings_file_path, "r") as settings_file:

        try:
            importlib.reload(settings)
        except BaseException:
            traceback.print_exc()
        # else:
        #     try:
        #         exec("from %s import %s as testcase_list_for_package" % (package_name, testcase_list_string))
        #     except BaseException:
        #         traceback.format_exc()


def run_with(args=None):
    """
    the args for func run_with(args=None) support below 3 types:
        1. None
            e.g: main.run_with()
        2. string which support below 4 types:
            1. tag_name
                e.g: main.run_with("smoke")
            2. testcase's class_name
                e.g: main.run_with("TestBackend_1")
            3. testcase_list_name
                e.g: main.run_with("tests_backend_testcases")
            4. package_name
                e.g: main.run_with("tests.backend")
            5. the string can be any combination of the above four kinds which separated by a whitespace
                e.g: main.run_with("smoke TestBackend_1 testcase_list_1 tests.backend")
        3. a list which element's type is String
            e.g:
                main.run_with(["TestCase_1", "tag_name", "tests.backend"])
              or
                main.run_with("TestCase_1 tag_name tests.backend".split())
              or
                main.run_with("TestAPITestcase".split())
    """
    print("====== init args:")
    print(args)

    if args is None:
        args = []
    elif isinstance(args, str):
        args = args.split()
    elif isinstance(args, list):
        for argv in args:
            if not isinstance(argv, str):
                raise TypeError(run_with.func_doc)
    else:
        raise TypeError(run_with.func_doc)

    # here remove the file name when execute run_test() from CMD
    if len(args) > 0:
        if args[0].__contains__(".py"):
            args = args[1:]

    print("init args after format:")
    print(args)

    # python3 run.py testcase_id=tc001 env=DEV thread=1 params="{'username':'lin','caseno': 50}" execution_id=10001 TESTCASE_TIMEOUT=10 DEBUG_RUN=False RERUN_FLAG=True
    update_config_str = ""
    params_str = None
    execution_id = None
    project_id = None
    case_id = None
    case_index = None
    callback_api = None

    priority = None

    for item in args:
        if item.find("=") != -1:
            item, value = item.split('=')[0].upper().lstrip().rstrip(), item.split('=')[1]
            print(item, "=", value)

            if item == 'ENV' or item == 'ENVIORNMENT':
                if value.lower() == 'dev':
                    update_config_str += 'ENVIRONMENT = Environment.DEV &'
                if value.lower() == 'qa':
                    update_config_str += 'ENVIRONMENT = Environment.QA & '
                if value.lower() == 'uat':
                    update_config_str += 'ENVIRONMENT = Environment.UAT & '
                if value.lower() == 'sit':
                    update_config_str += 'ENVIRONMENT = Environment.SIT &'
                if value.lower() == 'prod':
                    update_config_str += 'ENVIRONMENT = Environment.PROD &'

            elif item == 'THREADS' or item == 'THREAD' or item == 'QUEUE_SIZE':
                update_config_str += 'QUEUE_SIZE = %s &' % value
            elif item == 'PARAMS':
                params_str = value
                print(params_str)
            elif item == 'EXECUTIONID' or item == 'EXECUTION_ID':
                execution_id = value.lstrip().rstrip()
            elif item == 'PROJECTID' or item == 'PROJECT_ID':
                project_id = value.lstrip().rstrip()
            elif item == 'CALLBACK_API' or item == 'CALLBACKAPI':
                callback_api = value.lstrip().rstrip()
            elif item == 'CASEID' or item == "CASE_ID" or item == "TESTCASE_ID" or item == "TESTCASEID":
                case_id = value.lstrip().rstrip()
            elif item == 'CASEINDEX' or item == "CASE_INDEX":
                case_index = value.lstrip().rstrip()
            # elif item == 'PROJECTID' or item == "PROJECT_ID":
            #     project_id = value.lstrip().rstrip()
            elif item == 'PRIORITY':
                priority = value.lstrip().rstrip()
                priority = int(priority)
            else:
                update_config_str += "%s=%s &" % (item.lstrip().rstrip(), value)

    print("type(execution_id):", type(execution_id))
    print("execution_id:", execution_id)
    print("type(project_id):", type(project_id))
    print("project_id:", project_id)
    print("type(case_id):", type(case_id))
    print("case_id:", case_id)
    print("case_index:", case_index)
    print("callback_api:", callback_api)
    print("params_str:", params_str)
    print("type(params_str):", type(params_str))
    print("update_config_str:", update_config_str)

    update_config(update_config_str)


    if params_str:
        print("-------before:")
        print(type(BaseTestCase.TestEngineCaseInput))
        print(BaseTestCase.TestEngineCaseInput)
        BaseTestCase.TestEngineCaseInput = json.loads(params_str.replace("'", '"'))
        print("-------after:")
        print(type(BaseTestCase.TestEngineCaseInput))
        print(BaseTestCase.TestEngineCaseInput)

        print("====== BaseTestCase.TestEngineCaseInput:")
        print(BaseTestCase.TestEngineCaseInput)
        print("====== BaseTestCase.TestEngineCaseInput:")
        print(BaseTestCase.TestEngineCaseInput)

    if execution_id:
        BaseTestCase.TestEngineCaseInput["execution_id"] = execution_id

    if project_id:
        BaseTestCase.TestEngineCaseInput["project_id"] = project_id

    if case_id:
        BaseTestCase.TestEngineCaseInput["case_id"] = case_id

    if case_index:
        BaseTestCase.TestEngineCaseInput["case_index"] = case_index

    if callback_api:
        BaseTestCase.TestEngineCaseInput["callback_api"] = callback_api

    testcase_list = []
    testcases = []

    # executed doctor() to check the necessary configuration files, auto generate them if not exists
    doctor.doctor()

    try:
        importlib.reload(settings)
    except BaseException:
        traceback.print_exc()

    args_init = args
    args = []

    if case_id:
        args = [case_id]
    else:
        if priority == None:
            #说明命令行中没有传入priority=XXX参数，则走判断 是否有 tag的流程
            # 如果case_id is None，则说明 命令行没有传入 case_id=XXX,则走原来没有集成 test-engine的流程
            for arg in args_init:
                if arg.find("=") == -1:
                    args.append(arg)
        else:
            #说明命令行中有传入priority=XXX参数，则走根据 priority 执行的流程
            args.append("tests") #此处直接把所有的tests加入参数中，后面再根据priority过滤


    print(" ======== final args:")
    print(args)

    if len(args) < 1:
        usage()
        return

    elif len(args) >= 1:
        argument_deal_with_type_order = """
        for each parameter:
             first try to find it as a tag name,
             if no tag name found then try to find it as a TestcaseClassName,
             if no TestcaseClassName found then try to find it as a TestcaseList,
             if no TestcaseList found then try to find it as a package_name
        """
        print("the argument will be deal with by the follow order: %s" % argument_deal_with_type_order)
        try:
            auto_generate_testcase_list_from_csv.auto_generate_testcase_list_from_csv()
            testcase_dict_for_tags, testcase_dict_for_classname, testcase_dict_for_package = auto_generate_testcase_list.auto_generate_testcase_list()
            tags_list = []
            class_name_list = []
            package_list = []

            if project_id:
                # project_id 不为空 才执行如下操作
                for key in testcase_dict_for_tags.keys():
                    if len(key) > 0:
                        tags_list.append(key)
                for key in testcase_dict_for_classname.keys():
                    if len(key) > 0:
                        class_name_list.append(key)
                for key in testcase_dict_for_package.keys():
                    if len(key) > 0:
                        package_list.append(key)

                if len(package_list) > 0 or len(class_name_list) > 0 or len(tags_list) > 0:
                    print("packageList:")
                    print(package_list)
                    print("classNameList:")
                    print(class_name_list)
                    print("tagList:")
                    print(tags_list)

                    try:
                        headers = {
                            "Content-Type": "application/json",
                            "Host": "%s" % settings.TestEngineCallBackHost
                        }

                        callback_post_data = {
                            "id": "%s" % project_id,
                            "caseCount": len(testcase_dict_for_package["tests_testcases"]),
                            "packageList": package_list,
                            "classNameList": class_name_list,
                            "tagList": tags_list,
                        }

                        print("headers:")
                        print(headers)
                        print("callback_post_data:")
                        print(callback_post_data)

                        import requests
                        res = requests.post(
                            "%s/project/updateProjectCaseInfo" % settings.TestEngineCallBackAPI,
                            data=json.dumps(callback_post_data),
                            headers=headers)

                        print(res.json())
                    except BaseException:
                        traceback.print_exc()

            if case_id == 'auto_get_project_case_count_run_by_test_engine':
                print("-------=============-----------len(testcase_dict_for_package['tests_testcases']: %s" % len(testcase_dict_for_package["tests_testcases"]))
                print("will callback to test-engine with case_count & project_name")
                # todo: callback to test-engine with case_count & project_name, test-engine增加project时必须选择workflowId
                # 这样 传递 case_id=auto_get_project_case_count_run_by_test_engine & projectId=xxx的时候可以触发githubAction执行，此时需要同时
                # 传递 execution_id吗？ 注意 此 executoin_id是 d_project_case_count集合中的字段， 最好改成 fetch_id ?

                headers = {
                    "Content-Type": "application/json",
                    "Host": "%s" % settings.TestEngineCallBackHost
                }

                callback_post_data = {
                     # 此处 execution_id 其实是被执行的project的 projectId
                    # "projectId": "%s" % execution_id,
                    "projectId": "%s" % project_id,
                    "caseCount": len(testcase_dict_for_package["tests_testcases"]),
                }

                import requests
                res = requests.post(
                    "%s/projectCaseCount/create" % settings.TestEngineCallBackAPI,
                    data=json.dumps(callback_post_data),
                    headers=headers)


        except BaseException:
            traceback.print_exc()

        argument = args[0]

        if argument.lower() == "--help" or argument.lower() == "-help":
            usage()
            sys.exit()
        else:
            # the parameters for main.py can be any combination of the following 4 kinds which separated by a space:
            # 1. tag_name
            # 2. testcase's class_name
            # 3. testcase_list_name
            # 4. package_name
            #
            # e.g: execute CMD: "python main.py tag1 tag2 testcase_list1 testcase_list2 TestcaseClassName1 TestcaseClassName2 package_name_1"
            # all the testcase which fetched by the given parameters will be added into testcase_list, and will do
            # data deduplication for testcase_list
            #
            # for each parameter:
            #     first try to find it as a tag name, if no tag name found then try to find it as a TestcaseClassName,
            #     if no TestcaseClassName found then try to find it as a TestcaseList, if no TestcaseList found then
            #     try to find it as a package_name
            for argument in args:
                tag_name = None

                if argument == "quick_run":
                    continue  # todo: break? then quick_run can be the first argument, if break here, quick_run must be the last argument

                try:
                    if argument.__contains__("."):
                        print("****** Warning: there are '.' in tag {tag}, here replace '.' as '_' ".format(tag=argument))
                        tag_name = argument.replace(".", "_").lower()
                    else:
                        tag_name = argument.lower()

                    # first try to find the argument as a tag name
                    if isinstance(testcase_dict_for_tags[tag_name], list):
                        for testcase_for_tag in testcase_dict_for_tags[tag_name]:
                            testcase_list.append(testcase_for_tag)

                    elif issubclass(testcase_dict_for_tags[tag_name], base_testcase.BaseTestCase):
                        testcase_list.append(testcase_dict_for_tags[tag_name])

                    print("------ Found TAG: %s" % tag_name)

                except KeyError:
                    # it no tag name found from testcases then try to search it as a TestcaseClassName
                    try:
                        testcase_list.append(testcase_dict_for_classname[argument.lower()])

                        print("------ Found TestcaseClassName: %s" % argument)

                    except KeyError:
                        # if no TestcaseClassName found from testcases then try to search it as a TestcaseList
                        try:
                            if isinstance(testcase_dict_for_package[argument.lower()], list):

                                for testcase_for_package in testcase_dict_for_package[argument.lower()]:
                                    testcase_list.append(testcase_for_package)

                            print("------ Found TestcaseList: %s" % argument)

                        except KeyError:
                            try:
                                if argument.lower().__contains__("."):
                                    package_name = argument.lower().replace(".", "_") + "_testcases"

                                    if isinstance(testcase_dict_for_package[package_name], list):

                                        for testcase_for_package in testcase_dict_for_package[package_name]:
                                            testcase_list.append(testcase_for_package)

                                    print("------ Found Package: %s" % argument)

                                elif argument == "tests":
                                    package_name = argument + "_testcases"

                                    if isinstance(testcase_dict_for_package[package_name], list):

                                        for testcase_for_package in testcase_dict_for_package[package_name]:
                                            testcase_list.append(testcase_for_package)

                                    print("------ Found Package: %s" % argument)
                                else:
                                    print('''
****** ERROR: NO Tag/TestcaseClassName/TestcaseList/Package: '%s' Found in testcases!
Tips:
    If you are sure that there are no spelling errors, there are two possible reasons for not finding the corresponding case:
    1. The tag in the testcase script contains "ingore"
    2. run_test() is not implemented in the testcase script''' % argument)
                                    sys.exit()

                            except KeyError:
                                print('''
****** ERROR: NO Tag/TestcaseClassName/TestcaseList/Package: '%s' Found in testcases!
Tips:
If you are sure that there are no spelling errors, there are two possible reasons for not finding the corresponding case:
1. The tag in the testcase script contains "ignore"
2. run_test() is not implemented in the testcase script''' % argument)
                                sys.exit()

    if args[-1] == "quick_run":
        print("will execute by quick_run mode ...")

        testcase_module_class_list = []
        device_name_list = []

        testcase_dict_for_quick_run = {}

        for tc in testcase_list:
            tc_mddule_name = tc.__module__
            tc_class_name = tc.__name__

            if tc_class_name.__contains__("_real_device_"):
                testcase_module_class = tc_mddule_name.split("_real_device_")[0]
                testcase_module_class += "." + tc_class_name.split("_real_device_")[0]
                testcase_device_name = "real_device_" + tc_class_name.split("_real_device_")[1]

                if testcase_module_class not in testcase_module_class_list:
                    testcase_module_class_list.append(testcase_module_class)

                if testcase_device_name not in device_name_list:
                    device_name_list.append(testcase_device_name)

                testcase_dict_for_quick_run[tc_mddule_name + "." + tc_class_name] = tc

            elif tc_class_name.__contains__("_virtual_device_"):
                testcase_module_class = tc_mddule_name.split("_virtual_device_")[0]
                testcase_module_class += "." + tc_class_name.split("_virtual_device_")[0]
                testcase_device_name = "virtual_device_" + tc_class_name.split("_virtual_device_")[1]

                if testcase_module_class not in testcase_module_class_list:
                    testcase_module_class_list.append(testcase_module_class)

                if testcase_device_name not in device_name_list:
                    device_name_list.append(testcase_device_name)

                testcase_dict_for_quick_run[tc_mddule_name + "." + tc_class_name] = tc

        quick_run_list = []
        # here begin to get the quick_run_list by testcase_module_class_list & device_name_list
        index = 0
        for module_class_str in testcase_module_class_list:
            if index < len(device_name_list):
                device_name = device_name_list[index]

                module_name_str, class_name_str = functools.reduce(lambda x, y: x + "." + y,
                                                                   module_class_str.split(".")[:-2]) + "." + \
                                                  module_class_str.split(".")[-2] + "_" + device_name, \
                                                  module_class_str.split(".")[-1] + "_" + device_name

                quick_run_list.append(module_name_str + "." + class_name_str)

                index += 1
                if index == len(device_name_list):
                    index = 0

        # here begin to tran testcase_list to a dict
        quick_run_testcase_list = []
        for t in quick_run_list:
            quick_run_testcase_list.append(testcase_dict_for_quick_run[t])

        for tc in testcase_list:
            if not issubclass(tc, android_testcase.AndroidTestCase):
                quick_run_testcase_list.append(tc)

        testcase_list = quick_run_testcase_list
        # todo:  auto generate quick_run_testcase_list string list under tag_packages/ ?

    if isinstance(testcase_list, list):
        for testcase in testcase_list:
            if testcase not in testcases:
                testcases.append(testcase)

    elif issubclass(testcase_list, base_testcase.BaseTestCase):
        testcases.append(testcase_list)

    if testcases.__len__() == 0:
        print("************************ there are no testcases will be executed ************************")
        usage()
    else:
        # clean the execution environment
        if not settings.DEBUG_RUN:
            close_browsers_and_webdrivers()
        else:
            clean_appium_env()

        output_folder = project_info.output_folder
        begin_time = datetime.datetime.now()
        print("this execution's logs & screenshots under output_folder: %s" % output_folder)
        # QUEUE_SIZE is used to set how many test cases will be executed concurrently in settings.__init__.py
        # if not found, here set its default value as 1
        thread_pool_size = getattr(settings, "QUEUE_SIZE", 1)

        queue = collections.deque()

        include_android_testcase_flag = False
        non_backend_testcase_count = 0

        if priority:
            # 如果命令行参数中设置了 priority
            # fetch testcases and add into queue
            for testcase in testcases:
                if not issubclass(testcase, api_testcase.APITestCase):
                    non_backend_testcase_count += 1

                if issubclass(testcase, android_testcase.AndroidTestCase):
                    include_android_testcase_flag = True
                    device_name = TestCaseExecutor.get_device_name(testcase)

                    exec("TestCaseExecutor.%s = False" % device_name)

                # 只有大于或等于制定优先级的testcase才会被加入中执行队列中
                if testcase.priority is not None and testcase.priority >= priority:
                    print("add testcase:%s  ----  with priority:%s" % (testcase, testcase.priority))
                    queue.append(testcase)
        else:
            # fetch testcases and add into queue
            for testcase in testcases:
                if not issubclass(testcase, api_testcase.APITestCase):
                    non_backend_testcase_count += 1

                if issubclass(testcase, android_testcase.AndroidTestCase):
                    include_android_testcase_flag = True
                    device_name = TestCaseExecutor.get_device_name(testcase)

                    exec("TestCaseExecutor.%s = False" % device_name)

                queue.append(testcase)

        total_testcases_count = len(queue)
        testcase_executor_list = []

        if total_testcases_count == 0:
            print("\n\n\n++++++++++++++++++++++++++++ No testcases will be executed! ++++++++++++++++++++++++++++\n"
                                     "Please execute CMD 'python3 run.py --help' for more info :) ")
            return

        if non_backend_testcase_count > 0:
            # if non_backend_testcase_count <= thread_pool_size:
            #     thread_pool_size = non_backend_testcase_count

            import multiprocessing
            if thread_pool_size > multiprocessing.cpu_count() * 2:
                thread_pool_size = multiprocessing.cpu_count() * 2
                print("So many Non_Backend testcases concurrently run, here set the thread_pool_size = (2 * cpu_count:%s = %s)" % (
                        multiprocessing.cpu_count(), 2 * multiprocessing.cpu_count()))

        if total_testcases_count < thread_pool_size:
            thread_pool_size = total_testcases_count

        if include_android_testcase_flag is True:
            thread_pool_size = len(get_adb_devices())

        print("***" * 20 + " will start %s threads" % thread_pool_size)
        for i in range(thread_pool_size):
            testcase_executor_list.append(TestCaseExecutor(queue, total_testcases_count, output_folder, begin_time))

        for testcase_executor in testcase_executor_list:
            time.sleep(0.5)
            testcase_executor.start()

        for testcase_executor in testcase_executor_list:
            testcase_executor.join()

        if len(TestCaseExecutor.passed_testcases) > 0:
            print("****************** the Passed Testcases ClassName List(%s/%s): ******************" % (
                    len(TestCaseExecutor.passed_testcases), total_testcases_count))

        if len(TestCaseExecutor.failed_testcases) > 0:
            print("****************** the Failed Testcases ClassName List(%s/%s): ******************" % (
                    len(TestCaseExecutor.failed_testcases), total_testcases_count))

            failed_testcase_name_list = "%s" % os.linesep

            for tc in TestCaseExecutor.failed_testcases:
                failed_testcase_name_list += tc.__class__.__name__ + os.linesep

            print(failed_testcase_name_list)

        if not settings.DEBUG_RUN:
            close_browsers_and_webdrivers()
        else:
            clean_appium_env()


        try:
            if hasattr(settings, "generate_xunit_result") and settings.generate_xunit_result is True:

                xml_report.generate_xunit_result(TestCaseExecutor.passed_testcases,
                                                 TestCaseExecutor.failed_testcases,
                                                 output_folder,
                                                 begin_time
                                                 )
                print("generate xml report done...")
        except BaseException:
            traceback.print_exc()

            with open(output_folder + "error_log.txt", "w") as file_obj:
                file_obj.write(traceback.format_exc())
        finally:
            try:
                rerun_flag = False
                if hasattr(settings, "RERUN_FLAG"):
                    rerun_flag = settings.RERUN_FLAG

                if hasattr(settings, "SAVE_LOG_TO_DB"):
                    if settings.SAVE_LOG_TO_DB is True:
                        from . import database_helper

                        database_helper.save_execution_log(passed_testcases=TestCaseExecutor.passed_testcases,
                                                           failed_testcases=TestCaseExecutor.failed_testcases,
                                                           output_folder=output_folder,
                                                           project_name=project_info.project_name,
                                                           jenkins_job_name=TestCaseExecutor.jenkins_job_name,
                                                           rerun_flag=rerun_flag
                                                           )
            except BaseException:
                traceback.print_exc()
            finally:
                try:
                    html_report.Reporter(output_folder,
                                         TestCaseExecutor.passed_testcases,
                                         TestCaseExecutor.failed_testcases,
                                         TestCaseExecutor.error_testcases,
                                         begin_time,
                                         PLATFORM_INFO,
                                         True)
                except BaseException:
                    traceback.print_exc()

            # save the execution summary into DB (environment, total_execution_time,
            try:
                import lintest
                from . import database_helper
                if hasattr(settings, "SAVE_LOG_TO_DB"):
                    if settings.SAVE_LOG_TO_DB is True:
                        end_time = datetime.datetime.now()
                        execution_time = end_time - begin_time

                        database_helper.insert_execution_summary_log(
                            execution_id=output_folder.split(os.sep + "output" + os.sep)[1],
                            environment=settings.ENVIRONMENT,
                            os=PLATFORM_INFO,
                            automation_framework_version=lintest.__version__,
                            # total_execution_time=convert_to_seconds(execution_time),
                            total_execution_time=execution_time,
                            jenkins_job_name=None,
                            project_name=project_info.project_name,
                            total_testcases_count=total_testcases_count,
                            pass_testcases_count=len(TestCaseExecutor.passed_testcases),
                            fail_testcases_count=total_testcases_count - len(TestCaseExecutor.passed_testcases),
                            rerun_flag=1 if settings.RERUN_FLAG else 0
                        )
            except BaseException:
                traceback.print_exc()
            finally:
                pass

        print(os.linesep + "=*=*=" * 20 + os.linesep + "Execution Done! More details please see blow html report:" + os.linesep + "file:///" + output_folder + os.sep + "report.html")

        try:
            import webbrowser
            # webbrowser.open_new("file:///" + output_folder + os.sep + "report.html")
            # webbrowser.open_new("http://127.0.0.1:5000/")

        except BaseException:
            traceback.print_exc()

        if hasattr(settings, "COPY_REPORT_TO_SPECIFIED_PATH"):
            if settings.COPY_REPORT_TO_SPECIFIED_PATH is True:
                if not hasattr(settings, "SPECIFIED_REPORT_PATH") or not hasattr(settings, "SPECIFIED_REPORT_HOST_IP") \
                        or not hasattr(settings, "SPECIFIED_REPORT_HOST_PORT") or not hasattr(settings,
                                                                                              "SPECIFIED_REPORT_HOST_USERNAME") \
                        or not hasattr(settings, "SPECIFIED_REPORT_HOST_PASSWORD"):
                    raise Exception("如果 COPY_REPORT_TO_SPECIFIED_PATH = True, 则必须设置：SPECIFIED_REPORT_PATH & "
                                    "SPECIFIED_REPORT_HOST_IP & SPECIFIED_REPORT_HOST_PORT & SPECIFIED_REPORT_HOST_USERNAME & SPECIFIED_REPORT_HOST_PASSWORD")

                # # send report to server
                # def scp_report_to_specified_path(file_path=output_folder):
                #     ssh_client = paramiko.SSHClient()
                #     ssh_client.set_missing_host_key_policy(paramiko.AutoAddPolicy)
                #     ssh_client.connect(settings.SPECIFIED_REPORT_HOST_IP, settings.SPECIFIED_REPORT_HOST_PORT,
                #                        settings.SPECIFIED_REPORT_HOST_USERNAME, settings.SPECIFIED_REPORT_HOST_PASSWORD)
                #     scpclient = SCPClient(ssh_client.get_transport(), socket_timeout=15.0)
                #     # local_path = file_path
                #
                #     try:
                #         scpclient.put(file_path, settings.SPECIFIED_REPORT_PATH, recursive=True, )
                #     except FileNotFoundError as e:
                #         print(e)
                #         print("系统找不到指定文件" + file_path)
                #     else:
                #         print("文件上传成功" + output_folder)
                #     finally:
                #         ssh_client.close()

                # scp_report_to_specified_path(output_folder)


        #  todo: 需要再优化一下，save_log_to_db 不应该和 SAVE_SCREENSHOTS_PATH 耦合！！
        # if hasattr(settings, "SAVE_LOG_TO_DB"):
        #     if settings.SAVE_LOG_TO_DB is True:
        #         if not hasattr(settings, "SAVE_SCREENSHOTS_PATH"):
        #             raise Exception("如果 SAVE_LOG_TO_DB = True, 则必须设置：SAVE_SCREENSHOTS_PATH")
        #
        #         # only copy the screenshot for Non backend_testcases
        #         if non_backend_testcase_count > 0:
        #             import shutil
        #
        #             def ignore_pyc_files(dirname, filenames):
        #                 ig_list = []
        #                 for name in filenames:
        #                     if name.endswith('.pyc') or name.endswith(".log") or name.endswith("html") or name.endswith(
        #                             "xml"):
        #                         ig_list.append(name)
        #                 return ig_list
        #
        #             try:
        #                 # TODO: here should copy the screenshots to the dashboard server's static/screenshots
        #                 shutil.copytree(output_folder,
        #                                 settings.SAVE_SCREENSHOTS_PATH +
        #                                 output_folder.split(os.sep)[-1],
        #                                 ignore=ignore_pyc_files)
        #                 print("==== copy screenshots done ====")
        #                 print(settings.SAVE_SCREENSHOTS_PATH + output_folder.split(os.sep)[-1])
        #             except shutil.Error as e:
        #                 for src, dst, msg in e.args[0]:
        #                     # src is source name
        #                     # dst is destination name
        #                     # msg is error message from exception
        #                     print(dst, src, msg)


if __name__ == "__main__":
    close_browsers_and_webdrivers()
    run_with(sys.argv)
