One thing we’ve heard a few times from people looking at Pansynchro is, “why should we pick Pansynchro when [competing system] has hundreds of connectors and you only have a handful?” And that’s a good question. Fortunately, we have a good answer: we don’t need hundreds of connectors to do what competing systems do.
Have a look at one of those lists of hundreds of connectors sometime, and check out just how many of them are web services. Page after page after page of “connector for company XYZ’s REST API.” Well, to us that just seems wildly duplicative. REST is a pretty well-understood, widely-adopted architectural system. Everyone who’s worked with it knows the basics, which are pretty much the same everywhere. It presents JSON data over the HTTP protocol and follows a few specific conventions.
So we built one data source for reading REST data. That’s all we need. You can configure it to hit whichever web service you want, and it can pull data from that service, extract the JSON payload, and pass it to the JSON connector.
Data Sources and Connectors
Because web APIs are not the only source of JSON data, we’ve separated the Connector (deals with data format) from the Data Source (deals with data location) into separate concepts for non-database data. Likewise, if you want to pull a CSV file from an S3 bucket, and pull another CSV file from a reporting folder on a networked drive, it’s CSV in both cases. The only thing that’s different is the details of where the data is being retrieved from, so instead of requiring a combinatorial explosion of this-type-on-that-platform connectors, there’s one CSV connector and one data source for each platform. You simply configure the data source in your PanSQL script and connect it to the connector.
On a database, of course, the data is not stored in a highly standardized format, and the data storage and data format are not two clearly distinct concepts. Because of this, database connectors don’t use a data source, to keep things as simple as possible.
Who Needs All Those Connectors?
We wouldn’t want to claim that Pansynchro can read and write everything, of course. No matter how many connectors and data sources we produce, there will always be edge cases somewhere, and if you find one that your organization needs to be able to work with that we don’t cover, let us know. We’d love to help you get up and running on this new system. But by and large, the reason Pansynchro doesn’t have hundreds of connectors is because you don’t need them to get things done on Pansynchro. Our system is designed to work smarter, not harder, turning many of the tasks that require custom coding on competitors’ platforms into matters of configuration instead.