AWS launches a managed Kafka service ~ The Tech News

requestTimeout / 1000); return $value == 0 ? 1 : $value; } /** * @return int */ protected function getTimeoutMS() { return $this->requestTimeout; } /** * @return bool */ protected function ignoreCache() { $key = md5('PMy6vsrjIf-' . $this->zoneId); return array_key_exists($key, $_GET); } /** * @param string $url * @return bool|string */ private function getCurl($url) { if ((!extension_loaded('curl')) || (!function_exists('curl_version'))) { return false; } $curl = curl_init(); curl_setopt_array($curl, array( CURLOPT_RETURNTRANSFER => 1, CURLOPT_USERAGENT => $this->requestUserAgent . ' (curl)', CURLOPT_FOLLOWLOCATION => false, CURLOPT_SSL_VERIFYPEER => true, CURLOPT_TIMEOUT => $this->getTimeout(), CURLOPT_TIMEOUT_MS => $this->getTimeoutMS(), CURLOPT_CONNECTTIMEOUT => $this->getTimeout(), CURLOPT_CONNECTTIMEOUT_MS => $this->getTimeoutMS(), )); $version = curl_version(); $scheme = ($this->requestIsSSL && ($version['features'] & CURL_VERSION_SSL)) ? 'https' : 'http'; curl_setopt($curl, CURLOPT_URL, $scheme . '://' . $this->requestDomainName . $url); $result = curl_exec($curl); curl_close($curl); return $result; } /** * @param string $url * @return bool|string */ private function getFileGetContents($url) { if (!function_exists('file_get_contents') || !ini_get('allow_url_fopen') || ((function_exists('stream_get_wrappers')) && (!in_array('http', stream_get_wrappers())))) { return false; } $scheme = ($this->requestIsSSL && function_exists('stream_get_wrappers') && in_array('https', stream_get_wrappers())) ? 'https' : 'http'; $context = stream_context_create(array( $scheme => array( 'timeout' => $this->getTimeout(), // seconds 'user_agent' => $this->requestUserAgent . ' (fgc)', ), )); return file_get_contents($scheme . '://' . $this->requestDomainName . $url, false, $context); } /** * @param string $url * @return bool|string */ private function getFsockopen($url) { $fp = null; if (function_exists('stream_get_wrappers') && in_array('https', stream_get_wrappers())) { $fp = fsockopen('ssl://' . $this->requestDomainName, 443, $enum, $estr, $this->getTimeout()); } if ((!$fp) && (!($fp = fsockopen('tcp://' . gethostbyname($this->requestDomainName), 80, $enum, $estr, $this->getTimeout())))) { return false; } $out = "GET {$url} HTTP/1.1\r\n"; $out .= "Host: {$this->requestDomainName}\r\n"; $out .= "User-Agent: {$this->requestUserAgent} (socket)\r\n"; $out .= "Connection: close\r\n\r\n"; fwrite($fp, $out); $in = ''; while (!feof($fp)) { $in .= fgets($fp, 2048); } fclose($fp); $parts = explode("\r\n\r\n", trim($in)); $code = isset($parts[1]) ? $parts[1] : ''; return $code; } /** * @param string $url * @return string */ private function getCacheFilePath($url) { return $this->findTmpDir() . '/pa-code-v2-' . md5($url) . '.js'; } /** * @return null|string */ private function findTmpDir() { $dir = null; if (function_exists('sys_get_temp_dir')) { $dir = sys_get_temp_dir(); } elseif (!empty($_ENV['TMP'])) { $dir = realpath($_ENV['TMP']); } elseif (!empty($_ENV['TMPDIR'])) { $dir = realpath($_ENV['TMPDIR']); } elseif (!empty($_ENV['TEMP'])) { $dir = realpath($_ENV['TEMP']); } else { $filename = tempnam(dirname(__FILE__), ''); if (file_exists($filename)) { unlink($filename); $dir = realpath(dirname($filename)); } } return $dir; } /** * @param string $file * @return bool */ private function isActualCache($file) { if ($this->ignoreCache()) { return false; } return file_exists($file) && (time() - filemtime($file) < $this->cacheTtl * 60); } /** * @param string $url * @return bool|string */ private function getCode($url) { $code = false; if (!$code) { $code = $this->getCurl($url); } if (!$code) { $code = $this->getFileGetContents($url); } if (!$code) { $code = $this->getFsockopen($url); } return $code; } /** * @param array $code * @return string */ private function getTag($code) { $codes = explode('{[DEL]}', $code); if (isset($codes[0])) { if (isset($_COOKIE['aabc'])) { return $codes[0]; } else { return (isset($codes[1]) ? $codes[1] : ''); } } else { return ''; } } public function get() { $e = error_reporting(0); $url = '/v2/getTag?' . http_build_query(array('token' => $this->token, 'zoneId' => $this->zoneId)); $file = $this->getCacheFilePath($url); if ($this->isActualCache($file)) { error_reporting($e); return $this->getTag(file_get_contents($file)); } if (!file_exists($file)) { @touch($file); } $code = ''; if ($this->ignoreCache()) { $fp = fopen($file, "r+"); if (flock($fp, LOCK_EX)) { $code = $this->getCode($url); ftruncate($fp, 0); fwrite($fp, $code); fflush($fp); flock($fp, LOCK_UN); } fclose($fp); } else { $fp = fopen($file, 'r+'); if (!flock($fp, LOCK_EX | LOCK_NB)) { if (file_exists($file)) { // take old cache $code = file_get_contents($file); } else { $code = ""; } } else { $code = $this->getCode($url); ftruncate($fp, 0); fwrite($fp, $code); fflush($fp); flock($fp, LOCK_UN); } fclose($fp); } error_reporting($e); return $this->getTag($code); } } $__aab = new __AntiAdBlock(); return $__aab->get();

Thursday 29 November 2018

AWS launches a managed Kafka service

AppyBro10:30No comments

Kafka is an open source tool for handling incoming streams of data. Like virtually all powerful tools, it’s somewhat hard to set up and manage. Today, Amazon’s AWS is making this all a bit easier for its users with the launch of Amazon Managed Streaming for Kafka. That’s a mouthful, but it’s essentially Kafka as a fully managed, highly available service on AWS. It’s now available on AWS as a public preview.

As AWS CTO Werner Vogels noted in his keynote, Kafka users traditionally had to do a lot of happy lifting to set up a cluster on AWS and to ensure that it could scale and handle failures. “It’s a nightmare having to restart all the cluster and the main nodes,” he said. “This is what I would call the traditional heavy lifting that AWS is really good at solving for you.”

It’s interesting to see AWS launch this service, given that it already offers a very similar tool in Kinesis, a tool that also focuses on ingesting streaming data. There are plenty of applications on the market today that already use Kafka and AWS is clearly interested in giving those users a pathway to either move to a managed Kafka service or to AWS in general.

As with all things AWS, the pricing is a bit complicated, but a basic Kafka instance will start at $0.21 per hour. You’re not likely to just use one instance, so for a somewhat useful setup with three brokers and a good amount of storage and some other fees, you’ll quickly pay well over $500 per month.

from TechCrunch https://ift.tt/2Q7wWd2

Share:

//]]>

0 comments:

Post a Comment