Retrieving data from RM Server using Tibco Spotfire

SGolbert
SGolbert New Altair Community Member
edited November 5 in Community Q&A
Hi RMers!

I've been trying to retrieve a dataset from inside Tibco Spotfire using a RM web service. I've just retrieved the Titanic dataset and connected to the output port. Then in the web service I select Odata as output format and then JSON (I have tested all other variants as well).

However, I obtain an exception on the Spotfire side. 

An exception was thrown by the data connection.<br>Exception text:<br>System.NullReferenceException: Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.Execute(CancellationToken token, Action action)<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.Execute[T](CancellationToken token, Func`1 function)<br>   at Spotfire.Dxp.Data.Adapters.OData.ODataAdapterConnection.ListTablesCore(CancellationToken cancellationToken, ReadOnlyCollection`1& tables)<br>   at Spotfire.Dxp.Data.Access.Adapters.DataAdapterConnection.ListTables(CancellationToken cancellationToken, ReadOnlyCollection`1& tables)<br>   at Spotfire.Dxp.Data.Access.Adapters.DataAdapter.TryListTables(IDataAdapterExecutionContext executionContext, AdapterCredentialsValue credentials, CancellationToken cancellationToken, ReadOnlyCollection`1& tables)<br>   at Spotfire.Dxp.Data.Access.Adapters.DataAccessDataSourceLink.<>c__DisplayClass47_1.<ListTables>b__0(CancellationToken token)<br>   at Spotfire.Dxp.Data.Access.Adapters.DataAccessDataSourceLink.ExecuteAdapterMethod(ImpersonationPolicy impersonationPolicy, AdapterMethodDelegate method, Nullable`1 cancellationToken)
I have used user and password authentification, but that doesn't seem to be the problem. Could using anonymous login help? Has anybody tried something similar?

Thanks!
Sebastian

Tagged:

Best Answers

  • jwpfau
    jwpfau New Altair Community Member
    edited January 2019 Answer ✓
    Hi Sebastian,

    Have you tried to retrieve the data with another tool like Postman?

    It seems to fail during the retrieval of the Service Metadata Document
    http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?/$metadata
    A 404 page would also cause the NPE in Spotfire.

    Maybe don't use OData but just some generic JSON parsing.

    Greetings,
    Jonas
  • SGolbert
    SGolbert New Altair Community Member
    Answer ✓
    Hi all,

    I have used a R script in the end, but it was far easier that I thought:
    library("httr")<br>library("jsonlite")<br><br>fullDomain <- "http://server_ip:8080/api/rest/process/Titanic?"<br>r <- GET(fullDomain, <br>         authenticate("sgolbert", "My Password"))<br><br>document <- fromJSON(content(r,"text"))

    That actually brings the data to Spotfire, the rest is Spotfire specific. There is a workaround to make the user input user name and password and then encode it to base64, if you are concerned about security.

    Regards,
    Sebastian

Answers

  • jwpfau
    jwpfau New Altair Community Member
    Hi Sebastian,

    it's probably an authentication error, but to be sure enable logging in your TIBCO Spotfire client under "Help > Support Diagnostics and Logging" > Logging tab, set the log level to Debug. The log file should contain a better error message.

    Greetings,
    Jonas
  • SGolbert
    SGolbert New Altair Community Member
    Hi Jonas,

    I've done that and here is the log:
    2019-01-15T13:23:59,944+01:00 2019-01-15 12:23:59,944 sgolbert [Main Thread] INFO  Spotfire.Dxp.Forms.Application.DiagnosticsDialog [(null)] - The log4net log level (for the root logger) was set to DEBUG<br>2019-01-15T13:24:02,475+01:00 2019-01-15 12:24:02,475 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Leaving modal operation.<br>2019-01-15T13:24:02,506+01:00 2019-01-15 12:24:02,506 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Framework.Menu.UserActionRegistration [(null)] - Dispatched UserAction 'helpSpotfireDiagnostics' from sender 'S&upport Diagnostics and Logging'  to 1 handlers. Time elapsed: 00:00:19.0424008<br>2019-01-15T13:24:06,994+01:00 2019-01-15 12:24:06,994 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.Persistence.DisposableDataPoolSectionKey [(null)] - Creating new DisposableDataPoolSectionKey. Tag: EmbeddedData Value: 32b91b60-3ab1-4b03-aee1-ce978e46055d.<br>2019-01-15T13:24:06,995+01:00 2019-01-15 12:24:06,995 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.Persistence.DisposableDataPoolSectionKey [(null)] - Creating new DisposableDataPoolSectionKey. Tag: LinkedData Value: 0cd82d2c-7b19-4dda-a38a-3a35e319082f.<br>2019-01-15T13:24:06,997+01:00 2019-01-15 12:24:06,997 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.DataManager [(null)] - No save settings defined in document manager, creating new.<br>2019-01-15T13:24:06,998+01:00 2019-01-15 12:24:06,998 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.DataManager [(null)] - Default save settings are not applicable<br>2019-01-15T13:24:07,015+01:00 2019-01-15 12:24:07,015 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Entering modal operation.<br>2019-01-15T13:24:15,227+01:00 2019-01-15 12:24:15,227 sgolbert [48] DEBUG Spotfire.Dxp.Framework.HttpClient.HttpClient [(null)] - Create non Spotfire request http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?<br>2019-01-15T13:24:15,242+01:00 2019-01-15 12:24:15,242 sgolbert [48] DEBUG Spotfire.Dxp.Services.ServerInformation [(null)] - Setting server information; timeout: 3600000, keep alive: True<br>2019-01-15T13:24:15,399+01:00 2019-01-15 12:24:15,399 sgolbert [48] DEBUG Spotfire.Dxp.Internal.Utilities.Network [(null)] - Machine is part of domain: qs.intra<br>2019-01-15T13:24:15,477+01:00 2019-01-15 12:24:15,477 sgolbert [48] DEBUG Spotfire.Dxp.Services.ServerInformation [(null)] - Adding server to pre-authenticate: *.qs.intra<br>2019-01-15T13:24:15,524+01:00 2019-01-15 12:24:15,524 sgolbert [48] DEBUG Spotfire.Dxp.Framework.HttpClient.HttpClient [(null)] - Create Spotfire request to non spotfire server http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?<br>2019-01-15T13:24:15,820+01:00 2019-01-15 12:24:15,820 sgolbert [48] DEBUG Spotfire.Dxp.Framework.HttpClient.HttpClient [(null)] - Create non Spotfire request http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?<br>2019-01-15T13:24:15,820+01:00 2019-01-15 12:24:15,820 sgolbert [48] DEBUG Spotfire.Dxp.Framework.HttpClient.HttpClient [(null)] - Create Spotfire request to non spotfire server http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?<br>2019-01-15T13:24:16,023+01:00 2019-01-15 12:24:16,023 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Leaving modal operation.<br>2019-01-15T13:24:16,102+01:00 2019-01-15 12:24:16,102 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Entering modal operation.<br>2019-01-15T13:24:16,118+01:00 2019-01-15 12:24:16,118 sgolbert [ProgressOperationThread] DEBUG Spotfire.Dxp.Data.Access.Adapters.AuthenticationModeService [(null)] - Retrieving authentication mode for adapter 'Spotfire.ODataAdapter' (client type: 'DesktopClient').<br>2019-01-15T13:24:16,118+01:00 2019-01-15 12:24:16,118 sgolbert [ProgressOperationThread] DEBUG Spotfire.Dxp.Data.Access.Adapters.NullImpersonationPolicy [(null)] - Impersonating using current user: WindowsIdentity, Name: QS\sgolbert, AuthenticationType: Negotiate, ImpersonationLevel: None, IsAnonymous: False, IsAuthenticated: True, IsGuest: False, IsSystem: False, Token: 7148, Owner: S-1-5-21-2417176989-1276124730-1766474528-3983 (), User: S-1-5-21-2417176989-1276124730-1766474528-3983 ()<br>2019-01-15T13:24:16,133+01:00 2019-01-15 12:24:16,133 sgolbert [47] DEBUG Spotfire.Dxp.Framework.HttpClient.HttpClient [(null)] - Create non Spotfire request http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?/$metadata<br>2019-01-15T13:24:16,133+01:00 2019-01-15 12:24:16,133 sgolbert [47] DEBUG Spotfire.Dxp.Framework.HttpClient.HttpClient [(null)] - Create Spotfire request to non spotfire server http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?/$metadata<br>2019-01-15T13:24:16,290+01:00 2019-01-15 12:24:16,290 sgolbert [ProgressOperationThread] DEBUG Spotfire.Dxp.Data.Access.CancellationHelper [(null)] - System.AggregateException: One or more errors occurred. ---> System.NullReferenceException: Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.<br>   at Spotfire.Dxp.Data.Adapters.OData.ODataAdapterConnection.ListTablesCoreImpl(List`1 tablesBuilder)<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.<>c__DisplayClass1_0`1.<Execute>b__0()<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.<>c__DisplayClass2_0.<Execute>b__0()<br>   at System.Threading.Tasks.Task.Execute()<br>   --- End of inner exception stack trace ---<br>   at System.Threading.Tasks.Task.ThrowIfExceptional(Boolean includeTaskCanceledExceptions)<br>   at System.Threading.Tasks.Task.Wait(Int32 millisecondsTimeout, CancellationToken cancellationToken)<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.Execute(CancellationToken token, Action action)<br>---> (Inner Exception #0) System.NullReferenceException: Der Objektverweis wurde nicht auf eine Objektinstanz festgelegt.<br>   at Spotfire.Dxp.Data.Adapters.OData.ODataAdapterConnection.ListTablesCoreImpl(List`1 tablesBuilder)<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.<>c__DisplayClass1_0`1.<Execute>b__0()<br>   at Spotfire.Dxp.Data.Access.CancellationHelper.<>c__DisplayClass2_0.<Execute>b__0()<br>   at System.Threading.Tasks.Task.Execute()<---<br><br>2019-01-15T13:24:16,290+01:00 2019-01-15 12:24:16,290 sgolbert [ProgressOperationThread] DEBUG Spotfire.Dxp.Data.Adapters.OData.ODataAdapterConnection [(null)] - ListTables executed in 158 ms. NullReferenceException<br>2019-01-15T13:24:16,317+01:00 2019-01-15 12:24:16,317 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Leaving modal operation.<br>2019-01-15T13:24:16,331+01:00 2019-01-15 12:24:16,331 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Entering modal operation.<br>2019-01-15T13:24:34,209+01:00 2019-01-15 12:24:34,209 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Leaving modal operation.<br>2019-01-15T13:24:34,291+01:00 2019-01-15 12:24:34,291 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Entering modal operation.<br>2019-01-15T13:24:34,327+01:00 2019-01-15 12:24:34,327 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Forms.Application.FormsApplicationThread [(null)] - Leaving modal operation.<br>2019-01-15T13:24:34,329+01:00 2019-01-15 12:24:34,329 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Framework.DocumentModel.DocumentManager.Document [(null)] - Dispose document for Empty.<br>2019-01-15T13:24:34,331+01:00 2019-01-15 12:24:34,331 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Framework.DocumentModel.DocumentManager.Document [(null)] - Dispose commandHistory for Empty.<br>2019-01-15T13:24:34,332+01:00 2019-01-15 12:24:34,332 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Framework.DocumentModel.DocumentManager.Document [(null)] - Dispose embeddedDataSectionKey 32b91b60-3ab1-4b03-aee1-ce978e46055d for Empty.<br>2019-01-15T13:24:34,333+01:00 2019-01-15 12:24:34,333 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.Persistence.DisposableDataPoolSectionKey [(null)] - Decreasing reference count on key. Value: 32b91b60-3ab1-4b03-aee1-ce978e46055d. ReferenceCount before decreasing: 1.<br>2019-01-15T13:24:34,333+01:00 2019-01-15 12:24:34,333 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.Persistence.DataPool [(null)] - Clearing section for key 32b91b60-3ab1-4b03-aee1-ce978e46055d, Embedded.<br>2019-01-15T13:24:34,334+01:00 2019-01-15 12:24:34,334 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Framework.DocumentModel.DocumentManager.Document [(null)] - Dispose linkedDataSectionKey 0cd82d2c-7b19-4dda-a38a-3a35e319082f for Empty.<br>2019-01-15T13:24:34,335+01:00 2019-01-15 12:24:34,335 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.Persistence.DisposableDataPoolSectionKey [(null)] - Decreasing reference count on key. Value: 0cd82d2c-7b19-4dda-a38a-3a35e319082f. ReferenceCount before decreasing: 1.<br>2019-01-15T13:24:34,335+01:00 2019-01-15 12:24:34,335 sgolbert [Main Thread] DEBUG Spotfire.Dxp.Data.Persistence.DataPool [(null)] - Clearing section for key 0cd82d2c-7b19-4dda-a38a-3a35e319082f, Linked.<br><br>

    It is a NullPointer exception. I've been thinking that perhaps RM doesn't send the data in a compatible format.

    Regards,
    Sebastian
  • jwpfau
    jwpfau New Altair Community Member
    edited January 2019 Answer ✓
    Hi Sebastian,

    Have you tried to retrieve the data with another tool like Postman?

    It seems to fail during the retrieval of the Service Metadata Document
    http://rapidminer-vm.qs.intra:8080/api/rest/process/Titanic?/$metadata
    A 404 page would also cause the NPE in Spotfire.

    Maybe don't use OData but just some generic JSON parsing.

    Greetings,
    Jonas
  • SGolbert
    SGolbert New Altair Community Member

    I looked into it, getting the data as JSON it's possible by using a R or Python script. This is a bit more complex to what I had in mind, so I will just save the data as Excel or CSV.

    Thanks for the answer!
  • sgenzer
    sgenzer
    Altair Employee
    @land ?
  • SGolbert
    SGolbert New Altair Community Member
    Answer ✓
    Hi all,

    I have used a R script in the end, but it was far easier that I thought:
    library("httr")<br>library("jsonlite")<br><br>fullDomain <- "http://server_ip:8080/api/rest/process/Titanic?"<br>r <- GET(fullDomain, <br>         authenticate("sgolbert", "My Password"))<br><br>document <- fromJSON(content(r,"text"))

    That actually brings the data to Spotfire, the rest is Spotfire specific. There is a workaround to make the user input user name and password and then encode it to base64, if you are concerned about security.

    Regards,
    Sebastian
  • sgenzer
    sgenzer
    Altair Employee
    Glad you found a solution, @SGolbert. I would strongly encourage you to check out Sebastian's new Web Automation extension with its extensive JSON parsing tools. They are very good.

    Scott
  • SGolbert
    SGolbert New Altair Community Member
    I've read that they are good, but they are not free right? Is this coming to the core library someday?
  • sgenzer
    sgenzer
    Altair Employee
    @SGolbert no those tools are sold from Old World Computing (run by @land). TBH his pricing is VERY reasonable and well worth the $$ for what you get.

    Scott

Categories