cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Did you get called away in the middle of writing a post? Don't worry you can find your unfinished post later in the Drafts section of your profile page. X

How to configure windows native authentication that uses AD Domain to authenticate user for SQL DB

MA8731174
14-Alexandrite

How to configure windows native authentication that uses AD Domain to authenticate user for SQL DB

hello community,

  We dont have any username or password for our ms  sql company database, we use windows native authentication which uses Active Directory Domain to authenticate the user. How can i connect our microsoft SQL database ? I am unable to connect as i have nothing to write in database username and password. Please have a look on the picture below and let me know please how can i connect my sql database with thingworxs.....

 

MA8731174_0-1734002692827.png

 

20 REPLIES 20
MA8731174
14-Alexandrite
(To:MA8731174)

Hi @Surya_Tiwari do you have any idea?

Hello, try to add “ ;integratedSecurity=true” to your connection string.

Regards,
Constantine
MA8731174
14-Alexandrite
(To:Constantine)

thats already added in the end of connection string...

MA8731174
14-Alexandrite
(To:Constantine)

Connection String      :        jdbc:sqlserver:jamal.com;Database=cvs_absorber_test;integratedSecurity=true;

Hmm, this seems correct. What error do you get? In my understanding, in integratedSecurity mode MS SQL JDBC driver will try to authenticate the user who runs Tomcat process, so you need to ensure the it can access the database. If you run it as a Windows service, make sure this service executes under your "real" user, or just run it "manually".

 

You might also want to try removing username and password, just to see if that helps.

 

Finally, make sure sqljdbc_auth.dll is somewhere on the PATH, which Tomcat uses.

Please also check https://www.ptc.com/en/support/article/CS433191 and the used version of the JDBC driver, there was a breaking change.

MA8731174
14-Alexandrite
(To:Rocko)

@Constantine and @Rocko  Thanks for your replies. I ll try it out and update you whats a response. At the meantime i have one technical question in this usecase so i have 10 customers but the schema of data is same but due to privacy every customer who comes to mashup can only see his own data so there is a drop down with customers list and every customer who comes to mashup see his own name in the drop down and see his entries on the mashup he can also search the entry by putting serial number and search the entry.

He can also select the time duration and see the entries created during this time.

 

Now question is as i have connected sql with thingworx for these customers because in one day they create around 200 entreis so its a massive data and when they come to mashup they see entries which are latest 100 entries i stored in thingworx datatable and with timer event i delete from it old entries and always maintain 100 latest entries so that when customer comes on mashp it will load fast and he can see entries ..... on the sql side i have just one table which stored the entries of all customers. i have indexed field also doing some purposes now question is that is it a good approach? because i was also thinking that i should only use SQL even to load latest 100 entries but its not possible why? because of the big data in the table and when i fetch latest 100 entreis its fast but order by takes a lot of time to load the 100 entries like almost 15 seconds because i did test and created 500K entries for each customer in the database... so total 5 million entries for 10 customers.... 

 

what do you guys think is a best approach here which i am already doing with latest 100 entries on thingworx side or how?. because even i create a table for each customer on the SQL database still each table will reach to 500K entries in a year so a lot of data and as i mentioned showing latest 100 entries on the mashup is not fast.... please guide me in this.....

My understanding is you are using a DataTable to cache the last 100 entries from the SQL database table, because running the query directly against the database takes you 15 seconds.

You can do that, but I think it shouldn't be necessary. You should try to optimize query runtime by creating an index for the SQL table using customer id and date. Create the query and check in the execution plan that the index is used. It should be much faster that 15 seconds.

MA8731174
14-Alexandrite
(To:Rocko)

Thanks for your reply! Happy new year 2025!

 

Yes! you have understood correct that i cache latest 100 entries to show customer latest 100 entries as it would be fast when they come to mashup and see entries quickly. Now regarding to your suggestion about index, i have already made an index in SQL table but still the query takes a lot of time because by default it returns the oldest 100 entries with the query and then i have to sort them to show the latest 100 and this process takes 15 seconds there.

 

 

SQL TABLE

 

CREATE TABLE [ML1QualityDataTable] (
    [id] [INT] IDENTITY(1,1) NOT NULL,
    [serialNumber] VARCHAR(255) NOT NULL,
    [status] [BIT] NULL,
    [affix] VARCHAR(255),
    [lineLocation] VARCHAR(255) NOT NULL,
    [lineName] VARCHAR(255) NOT NULL,
    [userIdentification] [VARCHAR](255) NOT NULL,
    [qualityCheckTimestamp] Datetime2 NOT NULL,
    [jsonData] NVARCHAR(MAX), -- Column to store JSON data
    [productCategory] VARCHAR(255),
    CONSTRAINT [PK_ML1QualityData] PRIMARY KEY CLUSTERED ([qualityCheckTimestamp] ASC, [serialNumber] ASC)
);

 

 

INDEX

 

CREATE NONCLUSTERED INDEX idx_lineName_productCategory
ON dbo.ML1QualityDataTable (lineName, productCategory);

 

 

QUERY

 

--SELECT TOP 100 *
--FROM dbo.ML1QualityDataTable
--WHERE lineName = 'ML5'
--AND productCategory = 'PC5'
--ORDER BY qualityCheckTimestamp DESC;

 

So as you can see i can fetch the top 100 entries from sql but these entries are not sorted then.. for sorting i have to ORDER all entries first and as you can see in query that i am fetching the results by using my index.

 

Any Idea/suggestion in this regard would be helpful...

Hi @MA8731174 ,

You should really decouple things a bit. I will suggest something that does not require additional systems.

1. Create a SQL table with the purpose of a queue. ChatGPT will tell you how to do that (you'll need some SPs). This should always contain the top 10 entries, for each customer. A trigger should be used on the main table to add entries in this queue. This assumes you need the last 10 items that were added via INSERT. Don't use HTTP calls in the trigger - it probably won't scale.

2. Directly interrogate this table to get the entries you need. Other methods exist, but they are just too complex without benefits at this stage.

Would that work?

MA8731174
14-Alexandrite
(To:VladimirRosu)

Hi @VladimirRosu 

Thanks for your suggestions. I would like to give you in depth implementation information of my current store procedure. I send the request with a complete new entry with a serial number / timeStamp and it get saved in the SQL table. Then i go through steps of it  from front end so for example i have machine and i need to take picture on every step and send it to SQL database and my store procedure sees the if the entry exists with the serial number and timestamp it just stores subentries in the jsonData- NVARCHAR(MAX) column field in the SQL. 

 

So, we store main entry and then in one of its field jsonData we save its sub entries that are 7 or 10 in JSON... 

 

MA8731174_0-1737021541833.png

 

Now do you have any other idea to store the latest 100 entries ? other wise i have to copy this procedure exactly same and just add the condition to delete the other entries except the latest 100. is it a good approach what do you think? it means i have two similar procedures for two sql tables and then in one of them i have a condition to maintain only latest 100 entries..

or do you have any other recommendation..

 

 

In case you want to read my existing store procedure which is as below

 

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
-- SEGMENT 1: We start the store procedure for storing QualityData by initializing the column names we have in our table.
--
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
CREATE PROCEDURE InsertOrUpdateStatus
    @QualityCheckTimestamp Datetime2,
    @SerialNumber VARCHAR(255),
    @Status BIT,
    @Affix VARCHAR(255),
    @LineLocation VARCHAR(255),
    @LineName VARCHAR(255),
    @UserIdentification VARCHAR(255),
    @JsonData NVARCHAR(MAX),
    @StepNumber INT,
    @ProductCategory VARCHAR(255),
    @Comment VARCHAR(MAX)
AS
BEGIN
    SET NOCOUNT ON; -- I have used this because i dont want any updates from the SQL server to boost the PERFORMANCE for our system so that we dont get the message that how many rows are affected.
	--SET @QualityCheckTimestamp = CONVERT(datetime2, SWITCHOFFSET(@QualityCheckTimestamp, '+02:00'));
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
-- SEGMENT 2: We have declared these four variables here which we would use in store procedure so with IF NOT EXISTS i see that if entry with this serialNumber,lineName and QualityCheckTimestamp
--exists? If exists the we will NOT add it again and if the entry with this key does not exists then we will add the entry into our database.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
    DECLARE @CurrentJson NVARCHAR(MAX),
            @ExistingStatus BIT,
            @Index INT,
            @AnyStatusFalse BIT = 0

    IF NOT EXISTS (
        SELECT 1 FROM ML1QualityDataTable
        WHERE QualityCheckTimestamp = @QualityCheckTimestamp
        AND SerialNumber = @SerialNumber
        AND LineName = @LineName
    )
    BEGIN
        INSERT INTO ML1QualityDataTable
            (SerialNumber, Status, Affix, LineLocation, LineName, UserIdentification, QualityCheckTimestamp, JsonData, ProductCategory)
        VALUES
            (@SerialNumber, @Status, @Affix, @LineLocation, @LineName, @UserIdentification, @QualityCheckTimestamp, JSON_QUERY('[' + @JsonData + ']'), @ProductCategory)
    END
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
-- SEGMENT 2: FINISHED.
-- 
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------>
-- SEGMENT 3: Now we will see if the entry with that details exists then we will only edit its QualityStepData which is in database called jsonData. So I will just update the json object in that
-- jsonData column . I will update the status of the jsonData Object to true or false according to my wish. This requirement is implemented on the mashup. when i dont send any jsonData from backend
-- then it means that i am sending only the stepNumber to update its status. So this segment i can change the status of any object with the stepNumber i am sending from the backend.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------>
    ELSE
    BEGIN
        SELECT
            @CurrentJson = jsonData,
            @ExistingStatus = status
        FROM ML1QualityDataTable
        WHERE
            QualityCheckTimestamp = @QualityCheckTimestamp
            AND SerialNumber = @SerialNumber
            AND LineName = @LineName

        IF @StepNumber IS NOT NULL
        BEGIN
            IF @JsonData IS NULL OR LTRIM(RTRIM(@JsonData)) = ''  -- We have checked here that jsonData is not null and with LTRIM AND RTRIM i have removed the empty spaces so that we have jsonData.
            BEGIN                                                 -- I have here the jsonData null that means i will just update the status of the jsonData object which i have feature on the mashup.
                SELECT @CurrentJson = jsonData
                FROM ML1QualityDataTable
                WHERE QualityCheckTimestamp = @QualityCheckTimestamp
                AND SerialNumber = @SerialNumber
                AND LineName = @LineName

                IF ISJSON(@CurrentJson) = 1  -- ISJSON means if the input string is a right json format then it returns 1 otherwise if not then it returns 0.
                BEGIN
                    SELECT @Index = [key]   -- We will grab the key of json Object which has the stepNumber which i have sent from thingworx.
                    FROM OPENJSON(@CurrentJson)  -- OPENJSON converts the currentJson which is a jsonData into the rows format so that we can easily iterate on it
                    WHERE JSON_VALUE(value, '$.stepNumber') = CAST(@StepNumber AS NVARCHAR(255)) -- JSON_VALUE means we get value of the stepNumber from jsonData and iterate on it untill we get the equal stepNumber which i have sent from backend. CAST is used to set the type of stepNumber.

                    IF @Index IS NOT NULL   -- So if i get the index which is a key of the object i wanted so that i can manipulate the status of that object
                    BEGIN
                        SET @CurrentJson = JSON_MODIFY( -- JSON MODIFY updates the json and returns the updated results
                            @CurrentJson, --json data string we have 
                            '$[' + CAST(@Index AS NVARCHAR(255)) + '].status', --path of the object where we need to update the status
                            @Status -- the value which needs to be updated now
                        )
                        SET @CurrentJson = JSON_MODIFY(
                            @CurrentJson, --json data string we have 
                            '$[' + CAST(@Index AS NVARCHAR(255)) + '].comment',  --path of the object where we need to update the status
                            @Comment  -- the value which needs to be updated now
                        )
                        UPDATE ML1QualityDataTable
                        SET jsonData = @CurrentJson -- here we are setting the new jsonData Object which we have manipulated in that main entry.
                        WHERE QualityCheckTimestamp = @QualityCheckTimestamp
                        AND SerialNumber = @SerialNumber
                        AND LineName = @LineName

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
-- SEGMENT 3: ENDS
-- 
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
--SEGMENT 4: This part of the code is also for the mashup purposes which checks when we change the status of subentry from mashup and if we turn all the false entries into true then it will change
-- the main entry to true also and vice versa.
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
                        SET @AnyStatusFalse = CASE
                            WHEN EXISTS (
                                SELECT 1 FROM OPENJSON(@CurrentJson)
                                WHERE CAST(JSON_VALUE(value, '$.status') AS BIT) = 0
                            )
                            THEN 1
                            ELSE 0
                        END

                        UPDATE ML1QualityDataTable
                        SET Status = IIF(@AnyStatusFalse = 1, 0, 1)
                        WHERE QualityCheckTimestamp = @QualityCheckTimestamp
                        AND SerialNumber = @SerialNumber
                        AND LineName = @LineName
                    END
                    ELSE
                    BEGIN
                        PRINT 'No JSON object with the provided step number found.'
                    END
                END
                ELSE
                BEGIN
                    PRINT 'Current JSON data is not valid JSON.'
                END
            END

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
-- SEGMENT 4: FINISHED
-- 
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->

--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
-- SEGMENT 5: This part of the code is only to insert the new objects in the jsonData if i send the entry with the same primarykeys with additional jsonData object then i will just add that 
-- json Object into it.
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------->
            ELSE
            BEGIN
                SELECT @Index = [key]
                FROM OPENJSON(@CurrentJson)
                WHERE JSON_VALUE(value, '$.stepNumber') = CAST(@StepNumber AS NVARCHAR(255))

                IF @Index IS NOT NULL
                BEGIN
                    SET @CurrentJson = JSON_MODIFY(@CurrentJson, '$[' + CAST(@Index AS NVARCHAR(255)) + ']', JSON_QUERY(@JsonData))
                END
                ELSE
                BEGIN
                    SET @CurrentJson = JSON_MODIFY(@CurrentJson, 'append $', JSON_QUERY(@JsonData))
                END
                UPDATE ML1QualityDataTable
                SET jsonData = @CurrentJson
                WHERE 
                    QualityCheckTimestamp = @QualityCheckTimestamp 
                    AND SerialNumber = @SerialNumber
                    AND LineName = @LineName
            END
        END
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------>
-- SEGMENT 5: FINISHED
--
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------>

------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------>
-- SEGMENT 6: This will just update the general status of the main entry that means if any single of the status of the sub entry will be false then main entry would also be false otherwise 
--it will stay true.
------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------>

        -- Check if there is a need to update the main status based on the JSON content
        IF @JsonData IS NOT NULL AND LTRIM(RTRIM(@JsonData)) <> ''
        BEGIN
            DECLARE @ShouldUpdateStatus BIT;
            SELECT @ShouldUpdateStatus = CASE
                WHEN EXISTS (
                    SELECT 1 FROM OPENJSON(JSON_QUERY('[' + @JsonData + ']'))
                    WHERE CAST(JSON_VALUE(value, '$.status') AS BIT) = 0
                ) THEN 1
                ELSE 0
            END;

            IF @ShouldUpdateStatus = 1
            BEGIN
                UPDATE ML1QualityDataTable 
                SET status = 0
                WHERE 
                    QualityCheckTimestamp = @QualityCheckTimestamp 
                    AND SerialNumber = @SerialNumber
                    AND LineName = @LineName
            END
            -- No ELSE part to automatically update status to true
        END

    END

END

 

MA8731174
14-Alexandrite
(To:VladimirRosu)

I have tried an idea that executing the same SP which i have already to store the data with a new sql table and maintain only latest 100 entries so on every execution it checks if the entries are more then 100 then delete 1 oldest so it maintains the record.

 

I have added the deletion script below which is part of my store procedure. 

 

 WITH CTE AS (
        SELECT 
            id,
            ROW_NUMBER() OVER (PARTITION BY LineName ORDER BY QualityCheckTimestamp DESC, id ASC) AS RowNum
        FROM ML1QualityDataQueue
    )
    DELETE FROM ML1QualityDataQueue
    WHERE id IN (
        SELECT id FROM CTE WHERE RowNum > 5
    );

 

 

any comments on this approach?

So your index does not include the timestamp. This is what I meant. Either create a new index or add it to the existing one for qualitytimestamp desc. Compare the execution plan.

Also, if selecting and ordering does take 15 seconds, either the table is HUGE or something is off with the db. So your last idea to reduce tablesize I think is helpful.

MA8731174
14-Alexandrite
(To:Rocko)

My existing index on the table was this as below:

 

CREATE NONCLUSTERED INDEX idx_lineName_productCategory
ON dbo.ML1QualityDataTable (lineName, productCategory);

 

 

After adding qualityCheckTimestamp desc in the index

 

 

CREATE NONCLUSTERED INDEX idx_lineName_productCategory_qualityCheckTimestamp
ON dbo.ML1QualityDataTable (lineName, productCategory, qualityCheckTimestamp desc);

 

 

I execute this below query

 

SELECT TOP 100 *
FROM dbo.ML1QualityDataTable
WHERE lineName = 'ML1'
ORDER BY qualityCheckTimestamp DESC;

 

 

 

Entires are 10 million with this lineName and it took 1:20 minutes to get the latest 100 records.....

You could use an condition on your qualityCheckTimestamp to prefilter and reduce the amount of rows that need to be accessed, e.g. if you know the last 100 rows are always in the last 24h you could add that condition to the query. I agree with @VladimirRosu, you need a second table for that. With the current way, you are putting a huge load on the db again and again. Whenever you enter something into the large table, a trigger could enter the same data into a second small table which you monitor to have only 100 rows.

But this is beyond any relation to TWX, you might get more specific advice in a db/mssql forum.

MA8731174
14-Alexandrite
(To:Rocko)

Thanks @Rocko It works!

 

 

I made an index with lineName and qualityCheckTimestamp

--CREATE NONCLUSTERED INDEX IX_ML1QualityData_Fetch
--ON dbo.ML1QualityDataTable (lineName ASC, qualityCheckTimestamp DESC, id)
--INCLUDE ([serialNumber], [status], [affix], [lineLocation], 
--         [userIdentification], [jsonData], [productCategory]);

 

Query

SELECT TOP 100 *
FROM dbo.ML1QualityDataTable
WHERE lineName = 'ML1' 
AND  qualityCheckTimestamp >= DATEADD(DAY, -1, GETDATE())
ORDER BY qualityCheckTimestamp DESC;

 

It give me latest 100 entries from last 24 hours. This is what i wanted. The results i get in less then 1 second.!!

 

 

Thingworx side. (Why is such behaviour? Would you please let me know??) May be i am missing something on thingworx side

 

// It takes 30 seconds or longer
 SELECT TOP 100 * 
 FROM dbo.ML1QualityDataTable 
 WHERE lineName = [[lineName]] 
 AND  qualityCheckTimestamp >= DATEADD(DAY, -1, GETDATE())
 ORDER BY qualityCheckTimestamp DESC;

// When i give direct line name it fetches the data on the same speed as in sql server management like in less then 1 second.

SELECT TOP 100 *
FROM dbo.ML1QualityDataTable
WHERE lineName = 'ML8'
AND  qualityCheckTimestamp >= DATEADD(DAY, -1, GETDATE())
ORDER BY qualityCheckTimestamp DESC;

  

Can't really explain the last one, I would assume since [[lineName]] could be anything from a value to a subselect, the JDBC preparedstatement creates an execution plan that does not make use of the index.

 

But you can try <<lineName>> instead of [[lineName]] to see if it makes a difference,

In this case you need to make sure that lineName contains ONLY a line name and not some funky stuff like SQL injection.

MA8731174
14-Alexandrite
(To:Rocko)

 

working in less then 1 second same as on SQL management studio

MA8731174_0-1737360144996.png

 

not working

MA8731174_1-1737360174003.png

 

 

working but too slow like after 30 seconds

MA8731174_2-1737360212195.png

 

 

 

 

<<>> is plain text replacement so your argument probably needs to include the quotes. 'ML5'

MA8731174
14-Alexandrite
(To:Rocko)

Thank you so much!! :it works as expected i just add ' ' apostrophe around it. but would be good information for future about symbols around [[ ]]. 

 

SELECT TOP 100 * 
FROM dbo.ML1QualityDataTable 
WHERE lineName = <<lineName>>
AND  qualityCheckTimestamp >= DATEADD(DAY, -1, GETDATE())
ORDER BY qualityCheckTimestamp DESC;

 

i gave lineName like this as below:

MA8731174_0-1737361872372.png

 

Announcements


Top Tags