3 ways InfluxDB OSS Complements InfluxDB Cloud

As you may know, InfluxDB is available in multiple forms, including InfluxDB Open Source and InfluxDB Cloud. Customers will sometimes ask us, “If I’m using InfluxDB Cloud, is there any reason I should also consider using InfluxDB open source?”

It’s a fair question. InfluxDB Cloud has several capabilities not in InfluxDB OSS: InfluxDB Cloud is serverless so there are no servers to manage, it can automatically accommodate traffic surges, and we’re continually updating it to automatically deliver you the latest features.

We continually hear from customers that they prefer InfluxDB Cloud because it frees them from tedious database administration tasks so they can focus on their code. Even so, there are a few use cases where an InfluxDB Cloud customer can use InfluxDB Open Source to help manage their time series data. Let’s explore.

InfluxDB development environment

The other day, I drove my kid to go surfing in a remote spot on the California coast without cell coverage. While I watched him from the car, I was able to easily import sample data from a local CSV file into my local InfluxDB instance, then write some Flux code to query it. Coding from the beach — one of the silver linings of shifting to a remote work environment!

Because the InfluxDB UI, CLI, API, and client libraries between InfluxDB OSS and Cloud are practically identical, you can always shift resources like Flux scripts, InfluxDB Templates, InfluxDB Dashboards, Tasks, Alerts, and Grafana dashboards from OSS to Cloud (and vice-versa) with no modifications, facilitating a smooth DevOps pipeline.

The differences between InfluxDB Open Source and Cloud are rare, and highlighted in our documentation’s left-hand navigation.

We also make it easy to switch your Telegraf agents, client code using the InfluxDB client libraries, and InfluxDB CLI from pointing at a local InfluxDB open source instance, to an InfluxDB Cloud instance. Just change your InfluxDB Open Source URL from localhost:8086 to the appropriate InfluxDB Cloud URL, update your org, token, and bucket, and you’re good to go.

When using the CLI, you can create connection configurations, and easily switch between configurations as you move from Open Source to Cloud and back. And, we just added a -c flag to set your profile on the fly.

InfluxDB edge processing

In these situations, the edge instance of InfluxDB OSS often holds full-resolution telemetry for just the edge device, while the central InfluxDB Cloud instance holds telemetry across all devices. The degree to which InfluxDB Cloud’s telemetry is downsampled is often a function of business requirements, as well as connection reliability and bandwidth from the edge devices.

With this approach, folks responsible for managing the local environment — which would be a ferry, oil rig, farmland, etc. — have full telemetry to troubleshoot issues, without having to worry about connectivity to a central cloud instance. And folks in headquarters can analyze trends and issues across an entire IoT fleet using InfluxDB Cloud.

For example, one of our customers runs offshore oil rigs. All of the various time series metrics that they are gathering are stored in a local InfluxDB OSS instance and the operators of the rig have access to the information through dashboards. This information provides critical near real time visibility into what is occurring on the rig itself.

For some of the equipment being monitored, there is a desire to have access to this data — at a lower resolution than is being captured and stored locally — at the corporate headquarters. The corporate HQ folks can analyze the data from all of the rigs and look at the similarities and differences between the operational characteristics of the equipment in the aggregate. This allows for analyses like predictive maintenance to be performed and sent back to the rig operators.

Because InfluxDB OSS and Cloud share the same UI, API, and CLI, and both use Flux, teammates in the field and central office can easily share results and findings with each other.

To implement this pattern, you’ll need to dual-write to both InfluxDB Cloud and InfluxDB OSS to have a local backup. To do this, you’d put text similar to what’s below into your Telegraf configuration file (typically called telegraf.conf):

# Write to InfluxDB open source
[[outputs.influxdb_v2]] # For InfluxDB 2.0
urls = ["http://127.0.0.1:8086"]
token = "your-token"
organization = "your-organization-name"
bucket = "your-bucket-name"

# Write to InfluxDB Cloud
[[outputs.influxdb_v2]]
# Write to AWS Oregon instance.
# Other cloud URLs: https://v2.docs.influxdata.com/v2.0/reference/urls/
urls = ["https://us-west-2-1.aws.cloud2.influxdata.com"]
token = "your-token"
organization = "email-you-use-to-sign-into-influxdb-cloud"
bucket = "your-bucket-name"

With this configuration, every write goes to both your on-prem InfluxDB OSS instance and InfluxDB Cloud instance, at the same time. Because of this, data in both instances is kept in sync, up-to-the-second; there’s no batch process to periodically run backups.

Now that InfluxDB 0SS 2.0 is GA, look for us to make further engineering investments in this area of edge processing to simplify and support edge-to-cloud connection across these editions.

Air-gapped monitoring

If any of your servers or equipment are air-gapped, you can install InfluxDB OSS on them and use it to collect monitoring data in order to provide local observability and control. However, if you need higher levels of scalability or reliability than a single node of InfluxDB OSS can deliver, you should consider InfluxDB Enterprise. InfluxDB Enterprise provides horizontal scale-out and a high availability configuration.

So how does this usage of InfluxDB OSS complement InfluxDB Cloud? Because Cloud and OSS share the same API, CLI, web user interface, and client libraries, you can easily share assets like Dashboards, Tasks, and Alerts between your air-gapped InfluxDB OSS instances, and your InfluxDB Cloud instances. For example, if your company has standardized on Kubernetes, you build one set of k8s monitoring dashboards used at both your air-gapped and Internet-connected locations.

InfluxDB Cloud compared to InfluxDB OSS

Conclusion

We’re also grateful to our open source community of nearly 400 contributors to InfluxDB, who bring innovation to the shared InfluxDB Open Source and Cloud codebase. With them, we’re able to provide all of you with a range of options to suit your needs.

To get started with the patterns above, download InfluxDB Open Source and sign up for a free InfluxDB Cloud account. Then check out our getting started guide and feel free to engage our ever-helpful InfluxDB community, Slack, and GitHub repository for questions. Enjoy!

Occasional thoughts on tech, sailing, and San Francisco